The most serious defect of the studies of possible beneficial effects of water fluoridation was the lack of appropriate design and analysis. Many studies did not present an analysis at all, while others did not attempt to control for potentially confounding factors. Age, sex, social class, ethnicity, country, tooth type (primary or permanent), mean daily regional temperature, use of fluoride, total fluoride consumption, method of measurement (clinical exam or radiographs, or both), and training of examiners are all possible confounding factors in the assessment of development of dental caries.
While some of these studies were conducted in the 1940s and 50s, before the common use of such analyses, later studies also failed to use methods that were then commonplace. Many studies lacked any measure of variance for the estimates of caries presented. While most of the studies evaluating the proportion of children without caries contained sufficient data to calculate standard errors, only four of the eight studies that reported decayed, missing, and filled primary/permanent teeth provided any estimate of variance.
Outcomes measured and bias
The outcome of fluorosis was the most studied of all the adverse effects considered. Observer bias may be of particular importance in studies that assess fluorosis. Because assessment is subjective, unless the observer is blinded to the exposure status of the person being evaluated, bias can be introduced. Efforts to reduce potential observer bias were rarely undertaken in the included studies. The prevalence of fluorosis is overestimated by the indices used in the included studies because enamel opacities not caused by fluoride may be included. The degree to which the estimated 48% prevalence of fluorosis at a water fluoride concentration of 1 ppm overestimates the true prevalence is unknown. Figures and do not originate at 0% fluorosis because all areas included in the studies had at least a small amount of fluoride in the water. In addition, the effects of fluoride from other sources may also be playing a part.
Many studies of other potential negative effects also did not take steps to reduce bias or use analytic techniques to control for potential confounding factors. Interpretation of the results of these studies is difficult because few met inclusion criteria on each specific outcome and studies were generally of poor quality.
Statistical heterogeneity among studies may explain why individual studies report differing estimates of effect. Significant heterogeneity was found among studies of caries, fluorosis, and bone fracture and was also apparent among studies of cancer and other negative effects but could not be tested for. In addition, methodological and clinical diversity was present among these studies.
Publication bias is defined as the failure to publish research on the basis of the nature and directional significance of the results. Because of this, systematic reviews that fail to include unpublished studies may overestimate the true effect of an intervention. Because of the nature of the outcomes and study designs that we examined in this review we considered that the standard methods developed to investigate publication bias were not practical or appropriate. It is thus difficult to estimate whether publication bias is having an effect. As we took such a broad approach in searching for studies, any missed studies would have to be large and different from those that were included to overturn the overall result.
Given the level of interest surrounding the issue of public water fluoridation, it is surprising to find that little high quality research has been undertaken. As such, this review should provide both researchers and commissioners of research with an overview of the methodological limitations of previous research.
The evidence of a reduction in caries should be considered together with the increased prevalence of dental fluorosis. No clear evidence of other potential negative effects was found. This evidence on positive and negative effects needs to be considered along with the ethical, environmental, ecological, financial, and legal issues that surround any decisions about water fluoridation. Any future research into the safety and efficacy of water fluoridation should be carried out with appropriate methodology to improve the quality of the existing evidence base.
What is already known on this topic
Dental caries cause morbidity and suffering and incur costs
Artificial water fluoridation has been used as a community intervention to reduce the prevalence of dental caries for decades in some communities, but its use remains controversial
What this study adds
A systematic review of water fluoridation reveals that the quality of the evidence is low
Overall, reductions in the incidence of caries were found, but they were smaller than previously reported
The prevalence of fluorosis (mottled teeth) is highly associated with the concentration of fluoride in drinking water
An association of water fluoride with other adverse effects was not found