Patients and their families are increasingly turning to the Web as a source of medical information.15
In the last several years, there has been a tremendous growth in collaborative, Web 2.0 sites such as the online public encyclopedia Wikipedia, which enables any visitor to contribute and edit any article (although certain politically sensitive articles are not amenable to open editing, this does not apply to cancer-related information). In this study, we sought to determine how Wikipedia articles about five common and five uncommon cancers compared with articles about the same cancers from a peer-reviewed, expert-generated Web site, NCI's PDQ. The domains used for this comparison included depth of content, accuracy, discussion of controversial topics, and readability. In addition, the type and quality of references cited in Wikipedia articles, as well as the frequency with which these articles were edited, was assessed.
We found that although Wikipedia had similar accuracy and depth to the PDQ, the written style was more complex and thus might be less understandable to patients. We found no difference in the discussion of controversial topics between the two resources. Although the Wikipedia articles appeared to be more up-to-date, we acknowledge that this may possibly reflect a policy of PDQ not to discuss published studies until the pharmaceutical agents have been approved by the US Food and Drug Administration. Finally, regarding references in Wikipedia, more common cancer types had significantly more citations, as well as a higher proportion of citations from Medline-indexed articles.
To our knowledge, this is the most comprehensive and rigorous study comparing oncology articles between an expert-generated Web site and a wiki. Many parameters that are directly relevant to a patient's perspective, including accuracy, depth, and readability, were assessed. Weaknesses of the study include that its scope was limited to five common and five uncommon cancers, the fact that only a limited number of Web sites were examined, and the use of medically trained evaluators. In future studies, we intend to use a larger number of evaluators who are more representative of the general population.
Studies such as this one that assess the quality of online content will be ever more important as the Internet continues to increase in influence.16
Wikipedia is an especially prominent Web site; on general online searches for various medical terms and diseases, Wikipedia articles ranked among the first 10 results for 71% to 85% of the search engines and key words tested, surpassing professionally maintained Web sites (eg, National Institutes of Health MedlinePlus and National Health Service Direct Online).17
There have been very few studies assessing the quality of Wikipedia's medical articles. One study that focused on osteosarcoma found that Wikipedia was inferior to the NCI Web site.18
Another study that assessed the description of surgical procedures by Wikipedia found that although all the entries presented accurate content, 37.1% of articles had at least one critical omission. Interestingly, the study found a positive correlation between the frequency with which an article was edited and its accuracy.19
The latter finding partially concurs with our finding that articles about more common cancer types, which were significantly more frequently edited, had better quality references than those about uncommon cancers.
Our data indicate that although the informational content of articles on Wikipedia and PDQ is comparable, the former are less easy to read. The hypertext links provided by PDQ to a lay language dictionary further promote understanding, although this was not formally assessed in the readability metric. The implications of this disparity in reading grade are not known. Several studies have concluded that those patients who look for information online have above-average educations.1,17,20
However, many patients with cancer have impaired cognitive function.21
A complete understanding of the impact of disease, treatment, and educational attainment on the understanding and retention of Web-based information, although important, was beyond the scope of this study.
In conclusion, we found that Wikipedia and PDQ entries have comparable depth and accuracy, but the former were significantly less readable. On the basis of the sample articles tested, both appear to be reliable sources of information, but the editorial processes used by PDQ created a more readable result. Further research is required to ascertain what patient- and Web page–related factors determine optimal understanding and absorption of information. Such research will help in the design of the next generation of Web-based information systems.