This study is the first, to our knowledge, to have systematically evaluated websites providing information for patients about warfarin therapy. Specifically, this study has audited the quality, suitability and readability of the content of these websites to help gauge their utility for the general adult patient population including those with low literacy skills.34
The results of this study provide some important insights regarding medicines information on the internet, specifically information about warfarin therapy. Overall, the aspects of quality and suitability are adequate; the readability is generally poor and targeted toward patients with high skills.
This study found that the quality
of internet-based information about warfarin on most of the evaluated websites was generally adequate. These findings are consistent with the findings from previous studies,14
which have evaluated health information available on the internet for a range of different chronic diseases. This study also highlights that the quality of information about warfarin on the evaluated commercial websites is poor, which is also consistent with the findings of other studies.9
This is an important finding given the increasing reliance of patients on the internet as an information resource,1
as well as the increasing referral of patients by healthcare professionals to such websites. The relative advantages and disadvantages of non-commercial and commercial sites need to be carefully identified and communicated to patients, given that some commercial sites may not always be reliable sources of good quality information about warfarin.
Similar to the findings of a US-based study10
evaluating the suitability
of health information available on the internet about osteoporosis using the SAM instrument, the present study found that information about warfarin on most of the selected websites (including two commercial sites) was generally adequate (i.e. satisfactory) for the general adult population with limited literacy skills. Despite the overall adequate suitability ratings of information on these selected websites, specific deficiencies were identified regarding specific SAM criteria, such as graphics, layout and cultural appropriateness. This study's finding relating to the limited use of graphics/illustrations is consistent with those of other studies27
evaluating health information available on the internet. This is unfortunate given that these features help to effectively convey and define complex medical words and terminologies, and/or findings from clinical studies (e.g., risk versus benefit), thus having the potential to improve patient understanding of health information.45
In ethnically diverse countries, it is important to consider the cultural appropriateness of the information presented, given the ubiquitous nature of the internet making such information accessible to patients from a range of social, ethnic and cultural backgrounds.47
The present study highlights the issue that internet-based health information about warfarin does not always consider issues relevant to patients from non-mainstream ethnic groups, and/or how people from different ethnicities may interpret or apply the information. This reflects previous studies23
that have evaluated health information available on the internet about cancer therapy and which reported similar findings. Whilst it is difficult to cater to the needs of all existing socio-ethno-cultural groups, several key health websites have implemented simple measures to help meet the needs of their target populations; for example, the Canadian Breast Cancer Network (www.cbcn.ca) provides links to culturally relevant breast cancer information for aboriginal people, ethnic minorities and those for whom English is a second language. In regard to warfarin therapy, where complex information about lifestyle issues must be clearly communicated to patients (e.g., drug interactions with food/diet, risks of bleeding with normal activities of daily living), it is important to consider and address relevant socio-ethno-cultural ‘habits’ (e.g., diets, religious practices, health beliefs) within internet-based health information.
In regard to the readability,
this study highlights that the information presented on most websites is written at readability levels well beyond (e.g. grade 12) that of the average adult population. This result is consistent with Estrada et al (2000)25
and is important given that many patients receiving warfarin therapy are older patients with poor literacy skills. 25
For these patients, as well as others with poor literacy skills (e.g. poorly educated, culturally and linguistically diverse backgrounds), patient information about warfarin should be written at approximately school grade 8 or less to facilitate better understanding.6
Importantly, although a difference by approximately 2-4 grades was observed between the readability grades measured by the SMOG and F-K readability formulae, such a difference is not uncommon and is considered the result of variation between different measurement scales.24
Similarly, even though there is a disparity between the calculated reading grade levels for the manual and online SMOG formulae, they are all consistent with regard to the trends in increased reading grade levels required for the different websites. However, the comparatively higher readability grades generated by the online SMOG calculator compared to that of the manual SMOG formula warrant that care should be taken when using the online tools to measure the readability levels of health information available on the internet.
In summary, a wide variability in the quality, suitability and readability scores of internet-based health information about warfarin has been identified in this study. The overall scores indicate that whilst a website may score highly regarding quality parameters it may also achieve poor scores for other evaluated criteria, such as suitability and readability. In the current study, only www.anticoagulation.com.au
consistently attained higher scores/ratings in terms of the quality, suitability and readability of information abut warfarin.
Collectively, the study highlights that there are key areas for improvement to help increase the utility of the health and medicines information in relation to warfarin therapy. As a first measure, healthcare professionals might actively be aware of the information presented on websites, as well as purposefully identifying websites that patients may be accessing. By doing so, they will be able to not only identify misinformation but better direct their patients to more effective websites. Secondly, developers of internet-based health information could carefully consider each of these criteria and ensure that the information presented on their sites is relevant and suitable for their target audience (patient population) across each of the three criteria.
Limitations of the study
In interpreting the findings of this study, it is important to consider some of its potential limitations. Only English language sites were evaluated, and therefore the findings may not be generalisable to those websites written in other languages. The subjective nature of some quality and suitability criteria may potentially introduce variability in scoring, although a fair to good level of inter-rater consistency across the ratings was demonstrated. Furthermore, the SAM instrument principally evaluates the suitability of health information for the general adult population with limited literacy and it is not known to what extent this caters to other patient groups (e.g. older patients). The readability tools may have overestimated the required readability levels because they do not discriminate between commonly and infrequently used terms/words. For example, the analysis would not include commonly used, albeit polysyllabic, clinical and medical terms such as ‘warfarin’ and ‘anticoagulation'. Finally, a conflicting finding regarding the quality score/rating was measured by the HRWEF and the QCSS evaluation tools for the site www.warfarinfo.com
. However, such a finding may not be entirely unexpected given the different scoring/rating systems used and characteristics of evaluation criteria included in the above quality evaluation tools.