|Home | About | Journals | Submit | Contact Us | Français|
Objective To assess the quality of information and readability of the top internet sites for age-related macular degeneration (AMD).
Design An examination of the technical information provision, quality and readability of websites found during an internet search for ‘age-related macular degeneration’.
Setting Six internet search engines were used to find 26 unique sites on AMD.
Main outcome measures Technical information and quality were assessed using a simple grading system. Readability was assessed using a Simple Measure Of Gobbledygook (SMOG) rating.
Results Twelve organizational, seven academic and seven commercial sites were identified. The average technical scores were 82.3%, 67.9% and 65.2% for each type of site, respectively (P=0.097, one way ANOVA). The average quality scores were 62.2%, 62.6%, and 49.5% for each type of site, respectively (P=0.356, one-way ANOVA). The average SMOG ratings were 16.3, 16.1, and 16.2 for each type of site, respectively (P=0.983, one-way ANOVA). Fifteen of the sites provided details of new and emerging treatments, with seven providing a detailed discussion.
Conclusions Many websites are now meeting the challenge of providing comprehensive information about AMD and its new treatments. Quality scores were disappointing, with sites needing to provide more evidence of authorship and attribution of information. The majority of sites had SMOG scores above 10, making them difficult for the average person to understand. As physicians we need to help design and direct patients to sites that provide high quality, current information.
Age-related macular degeneration (AMD) is a common condition of the elderly affecting 155,000 new patients in the USA each year.1 The neovascular (or wet) form is responsible for the majority of associated visual loss. The emergence of new treatments for neovascular AMD means that patients and their relatives are keen to find out more to ensure they get the best visual outcome. Of particular interest are the anti-vascular endothelial growth factor (anti-VEGF) drugs, which are becoming first-line treatments in the USA1,2 but are not currently available on the NHS in England. Despite increasing coverage in the media, our patients had difficulty obtaining balanced information and were looking to the internet to find out more.
Information provided outside the verbal communication with the doctor can be helpful in improving knowledge and recall, and can aid patients in the decision making process.3-5 In 2002, Datamonitor reported that one-third of Europeans and half of Americans had used the internet for health information.6 In a study by the Kaiser Family Foundation in the USA,7 21% of those aged over 65 had been online to look up health information. For those aged 50-64 the internet had surpassed books and TV as a source of health information, although 46% did not trust the internet to provide accurate information. Whilst the internet is an important source of information which provides rapid access, it can be overwhelming and difficult for the non-specialist to know which sites provide current and correct explanations.8
There are no published papers on the quality and readability of AMD websites. We therefore conducted this study to assess the quality of information and readability of the top internet sites for AMD. We also determined whether sites were meeting the challenge of providing up-to-date information about new treatments. The websites were categorized into commercial, organizational and academic sites to see if any group had a particular bias. Commercial sites may be expected to focus narrowly on what they sell, whereas organizations are already providing patients with written information. Academic sites may be doing this but might also provide complicated technical information.9,10
The term ‘age-related macular degeneration’ (without quotation marks) was entered into six internet search engines on 22 September 2006, and the top ten sites identified by each search engine were reviewed within two weeks of this date. The internet search engines used were Google, Yahoo, MSN, Ask, Lycos and AOL. The sites were classified into three broad groups: commercial, organizational, or academic. Commercial sites were commercial health information sites or sites providing information purely for profit or sales (e.g. selling vitamins or vision aids). Organizational sites were government sites or not-for-profit organizations providing information, with occasional links to commercial entities. Academic sites were those sites providing information from hospital/educational institutions associated with universities.9,10
Technical information was assessed using a simple scoring structure we developed to look at information provided about macular degeneration (Table 1). The site content was compared with standard peer-reviewed information for diagnosis, risk factors, treatment for dry and wet forms of AMD, and other lifestyle advice. Incorrect information was noted and we documented whether the websites provided information on new and emerging treatments.
Site quality was assessed using a simple grading system developed by Martins et al.9 for elements deemed important for medical site evaluation based on the principles of Risk and Dzenowagis.11,12 The criteria include identification of ownership, purpose of site, authorship, author qualification, attribution of information, interactivity and currency (Table 2).
Readability was assessed using a Simple Measure Of Gobbledygook (SMOG) rating, as originally described by McLaughlin.13 This is a simple method to determine the reading level of written materials where the higher the score, the more difficult the text is to understand. If a person reads at or above a given level they will understand 90-100% of the information provided at that level. Most people will understand a text with a readability level of 10.
The commercial links provided by each search engine were also visited to assess what type of sites they linked to and whether they provided information on conventional or non-conventional therapies.
GraphPad Prism 4 statistical software was used for analysis. The technical and quality scores were converted to a percentage value of the maximum range, and the two were also combined to provide an overall score for each site. One-way ANOVA was used to compare the means for the three categories of website, with Bonferroni post test used for multiple comparisons. A finding was considered statistically significant at P<0.05.
The search identified 30 web addresses linking to 29 unique sites. Three sites were excluded from analysis because they did not provide clinical information about AMD. The remaining 26 links comprised seven academic, seven commercial and twelve organizational sites.
The mean quality score was 58.9% (range 23-100%). These were highest for the academic and organizational sites, with mean scores of 62.6% (range 31-100%) and 62.2% (range 38.5-92%) respectively. The commercial sites had lower quality scores—mean 49.5% (range 23-92%). These were not significantly different (P=0.356, one-way ANOVA).
The mean technical score was 73.8% (range 31-100%). The organizational sites had the highest mean scores at 82.3% (range 50-100%); the academic and commercial sites provided less information, with average scores of 67.9% (range 31-94%) and 65.2% (range 38-88%) respectively. The differences did not reach significance with one-way ANOVA testing (P=0.097; P>0.5 for all Bonferroni comparisons). Ten sites had a combined technical and quality score above 75%. Although the authors do not endorse the use of a particular website, the highest scoring sites for quality and technical information are provided in Table 3. None of the sites featured false or dangerous information.
The SMOG scores were high for the majority of sites, the mean being 16.2 (range 9-21), and were similar for each type of site (P=0.983, one-way ANOVA), with scores of 16.3 for organizational sites (range 9-20), 16.1 for academic sites (range 12-21) and 16.2 for commercial sites (range 13-21). Only one site, provided by Prevent Blindness America, scored less than 10.
Fifteen sites provided some information about anti-VEGF treatments. Seven of these provided detailed information about the drugs, including a brief summary of trial results and when one drug may be used over another. Four sites had links to pages giving more comprehensive information.
The commercial links yielded 28 different sites. Twenty-five of these were commercial sites and three were organizational. The commercial sites linked to eight sites providing conventional treatments (the majority for the purchase of recommended vitamins), eight sites providing alternative therapies (unproven supplements), and one site about an experimental treatment. The eight non-treatment sites were: two links to bookstores, two to other search engines, one to computer aids and one to a site selling spectacles. Two of these sites provided information about AMD, one requiring personal information to be given to a drug company before any health information would be released.
The challenge of providing current and comprehensive information about AMD is being met by many websites. There was no significant difference between the technical, quality and SMOG score for each category, but the organizational sites were most likely to provide information on new and emerging treatments. The quality scores were disappointing, with sites needing to provide more evidence of authorship and attribution of information. The majority of sites have high SMOG scores, indicating they may be difficult for the average person to understand.
The study was undertaken to simulate how a patient might look for information. It was limited to the first 10 sites identified by each search engine, as these are most likely to be visited by the consumer. The broad categories we used did not show any particular differences and may be related to the wide variety of organizations that can be found within each category. The strengths of the study include the use of an assessment of the readability of the site, along with a comprehensive technical and quality assessment. The technical assessment was measured against a range of background information for AMD. Whilst none of the sites assessed gave frankly incorrect or dangerous information, we did not make a more formal assessment of how treatments were discussed. Was the information balanced and in line with current practice, or was there bias towards particular treatments? Tools such as DISCERN can be used to assess sites aimed at helping patients to make treatment choices.14 We did not assess whether sites included tools to enlarge the text or change the background colour, features that are vital to people with poor vision. Another weakness of this study is that we did not assess how user-friendly the sites were or the ease of obtaining information. Other studies have tried to address this by measuring time or the number of mouse clicks required to obtain information.15 However, time is dependent on the speed of the internet connection and number of graphics, and the provision of a small amount of information on many pages may be easier to read and navigate than one page with all the information to be scrolled through. A study of website design attributes for retrieving health information by older adults found navigation/search usability, link usability, usefulness and colour to be important features.16
There are no previously published studies looking at the quality of information and readability of websites about AMD. Stone et al.17 performed a similar search in 1998 to assess whether sites featured conventional or unproven treatments. They found that 17 of 80 sites (21%) featured non-conventional treatment (those treatments that are not standard in peer-reviewed literature). However, some of these non-conventional treatments went on to become widely used. The management of AMD has changed dramatically since 1998, access to the internet is more widespread, and many patients wish to learn about the latest research developments and emerging treatments.
There are several scales to assess the readability of written information, but researchers rarely address the readability of websites.12,18 Graber et al.19 evaluated 50 health related websites and found the average reading level to be about 10th grade (equivalent to a reading age of 15-16 years), despite the fact the level of reading comprehension is lower in most patient populations. The top ranking websites we identified for quality and technological information (Table 3) all had SMOG ratings far greater than 10, the level most people would be expected to understand.13 Medical terms are long and may overinflate the SMOG score but this highlights the need to explain terms in simple language to enable a wider audience to appreciate the information provided. However, readability is just one aspect of reading comprehension and a low score may not provide sufficient depth for discussion of complex issues, particularly for those patients who have become familiar with the terminology.18
The quality of a site is not reflected in its search engine ranking. This is a complex process involving ease of categorization, popularity, number of links and the number of search engines it is listed on.12 The mean quality scores were disappointing for all three categories of site. Ownership was clearly stated for all the sites, and the majority gave a statement of purpose. However, less than half of the organizational and commercial sites provided information on authorship, author qualification, and attribution of factual information. One strategy to help the lay person find reliable sites is the use of quality assurance marking. The Health on the Net (HON) foundation is a not-for-profit organization that allows sites to display its trustmark if they meet eight acknowledged standards of quality. Seven of the sites we assessed displayed the HON accreditation, although it is important to note this is a self-certification scheme and two of these sites scored poorly (under 60%) on our quality assessment. Medical search engines, such as OMNI, can help target useful sites. However, a search performed on OMNI found that only three of their top ten sites provided information aimed at patients. The other links were for documents evaluating treatments and health economics that may be confusing or irrelevant to patients.
Our interest in this study was first stimulated by our patients' wish to learn more about the anti-VEGF drugs. These have become first-line treatment for wet AMD in the USA but are not yet available on the NHS. They are being talked about in the media and offered to many patients privately. Six months prior to starting this study we carried out a similar survey of websites for AMD and found 11 sites mentioning these treatments—eight providing detailed information about one of these drugs (pegaptanib sodium [macugen]), and two mentioning Ranibizumab (Lucentis) and Bevacizumab (Avastin). (Hannan, Presented at Royal Society of Medicine trainees meeting, June 2006.) Over the last six months many sites have begun to meet the challenge of providing information about these treatments. Fifteen sites now mention these treatments, with seven providing detailed information including summaries of trial results and discussion about all three drugs and when each might be used. The majority are provided by specialist organizations for patients with macular degeneration and blindness. Directing patients to these sites can be helpful in allowing them time to read and understand the information. However, this does not replace a detailed discussion and consent to treatment provided by the physician.
The commercial links to a search engine are gained by paying a fee and are motivated by sales and profit. These links revealed an even spread for the purchase of recommended vitamins,20 unproven supplements and non-treatment sites. These links are constantly changing and in our previous study we identified three sites recommending alternative treatments based on dubious evidence. Whilst some of the commercial links may be useful for patients, they and their physicians must be aware that some of the links are to unproven treatments. For patients desperate to prevent loss of vision, ‘non-official’ and alternative sites may be more readable and influential that the ‘official’ sites, leaving them vulnerable to misinformation.
Internet sites are providing widespread dissemination of specialist knowledge, challenging the balance of power between doctor and patient, and empowering patients to become more involved in their health-care decision making. Access to this information can help patients negotiate and lobby for new treatments.21 This is of particular relevance in England where there may be a substantial time lag between new treatments being widely used overseas (or privately) and their approval for use in the NHS by the National Institute for Clinical Excellence (NICE).
This study does not identify and recommend one particular site for patients to visit. Search engine results and websites change frequently, so physicians must be aware of the particular sites and information provided in their specialist area so they can direct patients to those sites that provide quality and current information. They must also take account of the ease of navigation and readability of a particular site. Although commercial sites may provide valuable information, most physicians would rather patients used organizational sites, which usually have an advisory board of specialists in the field, are experienced in providing information to patients, and are less likely to have ulterior motives in their presentations.10 We recommend that physicians get involved in the design and provision of technical information for the sites but also consider the readability of the text by involving lay people and using specifically designed toolkits. Further studies looking at how patients use the internet would help guide this. These could include understanding how sites are accessed and navigated, patients understanding of information provided, and how this affects uptake of treatments.
Competing interests The authors have no proprietary interest or research funding for the production of this work.
Contributorship CAR had the idea and did the searches, collated the data, analysed it, and wrote the paper. SH performed quality score for all sites. CAR, SH, CK and NM assessed technical and SMOG scores for a proportion of sites each. SH, CK and NM provided feedback for the idea and paper.