|Home | About | Journals | Submit | Contact Us | Français|
In the setting of declining U.S. literacy, new policies include use of clear communication and low literacy accessibility practices with all patients. Reliable methods for adapting health information to meet such criteria remain a pressing need.
To report method validation (Study 1) and method replication (Study 2) procedures and outcomes for a 5-step method for evaluating and adapting print health information to meet the current low literacy criterion of <5th grade readability.
Sets of 18 and 11 publicly-disseminated patient education documents developed by a university-affiliated medical center.
Three low-literacy criteria were strategically targeted for efficient, systematic evaluation and text modification to meet a <5th grade reading level: sentence length <15 words, writing in active voice, and use of common words with multisyllabic words (>2–3 syllables) minimized or avoided. Inter-rater reliability for the document evaluations was determined.
Training in the methodology resulted in inter-rater reliability of 0.99–1.00 in Study 1 and 0.98–1.00 in Study 2. Original documents met none of the targeted low literacy criteria. In Study 1, following low-literacy adaptation, mean reading grade level decreased from 10.4±1.8 to 3.8±0.6 (p<0.0001), with consistent achievement of criteria for words per sentence, passive voice, and syllables per word. Study 2 demonstrated similar achievement of all target criteria, with a resulting decrease in mean reading grade level from 11.0±1.8 to 4.6±0.3 (p < 0.0001).
The 5-step methodology proved teachable and efficient. Targeting a limited set of modifiable criteria was effective and reliable in achieving <5th grade readability.
Low health literacy, a persisting public health challenge in the U.S., is linked to poorer health status, increased morbidity and mortality, and higher healthcare costs.1,2 The most recent National Assessment of Adult Literacy (NAAL) revealed that only 12% of American adults demonstrate proficient literacy with written materials.3,4 Although lower literacy among racial and ethnic minorities has been found to contribute to health disparities,5,6 the NAAL revealed a population decline in literacy over the past ten years, with greatest decline among White adults and persons with a high school diploma, GED, or some vocational education.3,4
These findings suggest that lower literacy impacts the U.S. adult population broadly, warranting population-level intervention. A universal precautions approach has been recommended;7 recent federal legislative action mandates use of clear, plain language in written documents for the public, across government sectors.8
The current, low-literacy criterion for readability of health information is <5th grade,9–11 yet most patient education materials and websites remain at 12th grade or college level readability.12,13 Resources are available to assist with awareness of health literacy recommendations and materials adaptation (See Appendix A). However, an efficient, systematic approach that consistently and reliably results in <5th grade readability, especially while preserving essential medical terms, remains a need for healthcare organizations and professionals.14,15
This paper describes a 5-step approach for essential evaluation and efficient adaption of print patient education text to meet <5th grade readability. We report: a) a method validation study (Study 1) to test effectiveness of the 5-step approach in achieving targeted low-literacy criteria in a set of 18 publicly-disseminated patient education documents, and b) a method replication study (Study 2) to test reliability of the personnel training and consistency of the 5-step approach in achieving targeted literacy criteria in an additional set of 11 patient education documents.
Criteria for low-literacy patient education materials were derived from the Centers for Disease Control and Prevention (CDC),9 National Institutes of Health/National Cancer Institute (NIH/NCI),10 and Doak and colleagues.11 Of 32 consensus criteria available on literacy demand, formatting, and behavioral activation characteristics14 (See Appendix B), the following literacy-demand criteria were selected for efficiency and effectiveness in adapting text to meet a <5th grade reading level: 1) sentence length <15 words; 2) writing in active voice (<5% passive voice); and 3) use of common words, with multisyllabic words (>2–3 syllables) minimized or avoided. These criteria were deemed modifiable, and reading grade level, sentence length and passive voice can be readily evaluated using available electronic resources. (See Hill-Briggs and Smith14 for descriptions of modifiable low-literacy criteria, operational definitions, and procedures.)
The 5-steps in the approach are shown in Table 1. Importantly, the approach begins with comprehensive evaluation to determine adaptation needs at the sentence level of analysis. For the two studies reported herein, readability statistics available in Microsoft Word, 2003/2007 were used to evaluate and adapt the documents. The Flesch-Kincaid Grade Level test rates text on a U.S. school grade level, with a score of 5.0 indicating that an average fifth grader can understand the document.16
Following document evaluation, low-literacy adaptation was approached strategically, targeting those sentences that were at or above a 5th grade reading level, and the specific elements of the evaluated sentence that were not within the low-literacy targets. Specific adaptation methods included use of common, familiar words; use of resources (e.g. thesaurus) for word and phrasing alternatives; use of a simple, direct, and straightforward manner for sentence wording; expressing one main thought at a time; refraining from adding multiple phrases, clauses, or modifiers within a sentence; use of formatting changes such as bullets or tables when appropriate; avoidance of scientific jargon or medical terminology that was above the target readability; and consistency in use of the chosen words or phrases used to substitute for frequently occurring medical language. When medical terminology that exceeded the readability level was necessary, those technical, concept, category, or value judgment terms were introduced with an understandable explanation or example, prior to introduction of the scientific terminology. The scientific term, rather than the explanation, was placed in parentheses. This allows the reader to encounter the understandable explanation in the course of the text, with the scientific term treated as parenthetical to the explanation. Following low-literacy adaptation, the adapted documents underwent evaluation of readability criteria, and the readability statistics of the original and literacy-adapted documents were compared.
The 5-step methodology was tested using a set of 18 publicly disseminated patient education documents covering a variety of diabetes and lifestyle topics, developed by a university-affiliated medical center. Three research assistants (high school, bachelor’s and master’s level education, respectively, with no prior experience in low literacy evaluation, writing, or document adaptation) were hired and trained as literacy adaptation specialists for the study. Assistants received an initial 2-hour in-person training in the 5-step methodology prior to working with the documents. They then received a 1-hour supplemental training, during which instruction and materials were provided to address specific challenges in the treatment of medical and legal terminology, and supplemental material in documents (e.g. resource lists). A combined literacy adaptation manual was developed for training new personnel, with the following content: Performance of the 5-Step Method, Additional Exclude/Include Criteria for Readability Evaluations, Treatment of Medical Terminology, Legal Content, Tip Sheet, and Demonstration of Method(s) and Practice. Each assistant received supervision to ensure quality of the adaptation and consistency of content with the original document. Each document was evaluated by two assistants for determination of inter-rater reliability.
Study 2 was conducted using a set of 11 publicly disseminated patient education documents from the same original source. Two different literacy evaluators/adapters (bachelor’s level education, with prior editorial experience, but without formal training in low literacy evaluation and adaptation) participated in this study. They completed the comprehensive 2-hour total training, using the manual created in Study 1.
Inter-rater reliability for document-level literacy statistics was assessed using Pearson product-moment correlation coefficients. Inter-rater reliability was determined for original and adapted documents. Document-level literacy statistics of the original and adapted documents were compared using paired-samples t-tests to determine statistical significance of mean improvements in literacy variables. Analyses were conducted using PASW Statistics 18, Release Version 18.0.0 (© SPSS, Inc., 2008, Chicago, IL).
Following the basic training, inter-rater reliability for document-level readability evaluation ranged from 0.81 – 0.99 for the set of original documents. After adding the 1-hour supplemental training, inter-rater reliability of the literacy evaluations increased to 0.99 – 1.00 for each pair of evaluators, and for each original document. Inter-rater reliability remained at 0.99 – 1.00 for adapted documents.
Readability statistics for the documents are shown in Table 2. Mean reading grade level of the original documents was 10.4 ± 1.8, with no documents meeting low-literacy criteria investigated. Following adaptation, mean reading grade level of the documents was 3.8 ± 0.6, with each document meeting the < 5th grade criterion. All adapted documents met targets for passive voice and number of words per sentence. Table 3 illustrates an original document excerpt and the literacy-adapted version of that excerpt. Rephrasing of text, shortening of sentences to simple sentence constructions, and word substitutions were the most commonly used text adaptations in the set of 18 documents, while the most commonly used format changes were bulleting and tables, particularly for content requiring medical terminology (i.e. medication lists and descriptors).
Syllables per word was compared with the number of characters per word readability statistic (Table 4). Both statistics improved with document literacy adaptation. The use of common, familiar words in the adaptation process resulted in a decrease in mean percentage of words with >3 syllables from 8.3% ± 4.9 in the original documents to 2.5% ± 2.2 in the adapted documents.
Readability statistics are shown in Table 5. After 2-hour comprehensive training, inter-rater reliability for literacy evaluation of original documents was 0.99 – 1.00, and for adapted documents, 0.98 – 1.00. Mean reading grade level of the original documents was 11.0 ± 1.8., and no original documents met target low-literacy criteria. Study 2 yielded equally successful adaptations as Study 1 (p-values <0.0001).
The 5-step methodology, which relies on commonly available electronic readability tools, demonstrated effectiveness as an evaluation and adaptation approach to meet the <5th grade readability criterion for public health information. Effectiveness of the approach was demonstrated in both the method validation and replication studies. Personnel with no prior experience were able to be trained to perform the method successfully.
Multiple techniques, used in combination, were needed to meet targeted readability statistics. Mere word substitution was not effective as an overall technique for document adaptation; it was prone to inaccurate or awkward sentence constructions, and it often did not reduce sentence-level readability sufficiently. Rephrasing was necessary to preserve meaning and to introduce essential medical terms, while still reducing literacy demand. All instances of passive voice required rephrasing. Changing passive voice phrasings (e.g. Eating foods that have fewer calories is used to help with losing weight) to declarative statements (e.g. Eat foods with fewer calories to help lose weight) not only reduced reading grade level but made the messages more actionable for the reader. Further, we found that meeting the <5th grade criterion was realistic, and reading levels as low as 2nd and 3rd grade were achievable without using presentation styles or language that might be perceived as more appropriate for children than adults.
Our method utilized the Flesch-Kincaid formula to assess reading grade level. It is important to note additional available readability formulas that are widely used, including the FOG Index, the Simple Measure of Gobbledygook (SMOG), and the Fry readability graph. Although these formulas are each calculated differently, they have been found to correlate highly with each other, and there has been no evidence to support increased accuracy of one over another in determining reading grade level.17,18 All four formulas can be calculated manually; the Flesch-Kincaid formula has an accessible electronic calculation, which provides the benefit of time efficiency and practicality in routine use and with large numbers of documents. A limitation of using the Microsoft Word automated readability statistics is that the syllables per word statistic is not provided; characters per word is calculated. This study evaluated number of syllables per word using the Doak11 method, and a comparison table, with number of characters per word, is provided. During the adaptation process, multisyllabic words were readily identified without a formal calculation. The process of using common, more familiar words automatically resulted in fewer syllables at the sentence and document levels of analysis.
With regard to literacy adaptation content evaluation, the articles did not undergo a process of pre-/post- evaluation for preservation of meaning as part of research. However, after completing literacy adaptation, the adapted articles were reviewed alongside the original articles, by the institutional business office responsible for materials development and copyright. The business office approved the literacy-adapted articles as consistent in content and meaning. The articles then underwent review by representatives of a national healthcare organization which, after reviewing the articles, requested use of the full sets of literacy-adapted (but not the original) articles for dissemination as routine patient education materials to their members/clients. Although data were not collected during these reviews, both professional entities determined the literacy-adapted materials to be of high quality and value, consistent with or exceeding that of their non-literacy-adapted counterparts.
Testing acceptability and effectiveness of the materials in educating the public is ultimately needed to evaluate utility of the 5-step method. The method has been used to adapt patient education materials in previously reported clinical outcome studies, with findings of high satisfaction and ease of understanding ratings by adults with literacy as low as ≤3rd grade as well as those with average literacy,19 and by persons with mild cognitive and/or visual impairment,20 another subgroup for whom clear communication is particularly necessary. Intervention studies using this literacy-adaptation method have resulted in increased patient knowledge19,21 and improved behaviors and disease control21 in lower-literate samples.
There are directions deserving of further research and development. First, the 2-hour training, which was efficient and reliable, warrants testing in alternative instruction modes (e.g. web-based, electronic media such as CD-Rom). Second, in developing and testing the 5-step method, consistent “rules” for reducing literacy demand emerged for several elements of language (e.g. treatment of numbers, contractions, sentence structure for active voice) that may be amenable to software programming. Third, there is a need to evaluate the efficacy of the 5-step process in other languages. The overall framework for a systematic approach to low literacy adaptataion is transferrable, as are some elements of formatting. However, we do not expect that the specific literacy demand criteria will hold across languages and cultures. Effective low literacy text modification is based upon understanding of the language construction, parts of speech, characters utilized (length, letter pairings, etc.), and meaning/messaging, all of which determine how information is processed, stored, and learned. The neuropsychology literature reveals significant differences in the structure, processing and comprehension of Spanish language, for example, such that the English language low-literacy criteria may not be directly transferrable. Low-literacy adaptation in Spanish, and other languages, does require examination of those features of the respective language itself.
Finally, the problem of declining literacy in the U.S. over the past decade is one of great importance that likely requires both educational and political reform to reverse. The methodology described herein allows health information to be accessible to the growing proportion of the population with basic to below basic literacy; this is essential for improving patient self-management and health outcomes.22–25 However, meeting this current demand should not obfuscate the concurrent need for actions directed toward stopping and reversing the low literacy trends in the U.S.
This research was supported by NIDDK Diabetes Research and Training Center grant P60 DK079637 (O.D.) and NHLBI grant R01HL08975 (F.H-B., K.S.). Data were presented in part at the Annual Health Literacy Conference, Washington, DC, October 2009.