|Home | About | Journals | Submit | Contact Us | Français|
To perform a systematic review of interventions designed to improve health outcomes for persons with low literacy skills.
We searched MEDLINE, Cumulative Index to Nursing and Allied Health (CINAHL), Educational Resources Information Center (ERIC), Public Affairs Information Service (PAIS), Industrial and Labor Relations Review (ILLR), PsycInfo, and Ageline from 1980 to 2003.
We included controlled and uncontrolled trials that measured literacy and examined the effect of interventions for people with low literacy on health outcomes, including health knowledge, health behaviors, use of health care resources, intermediate markers of disease status, and measures of morbidity or mortality. Two abstractors reviewed each study for inclusion. Disagreements were resolved by consensus among the research team.
One reviewer abstracted data from each article into an evidence table; the second reviewer checked each entry. Disagreements about information in evidence tables were resolved by team consensus. Both data extractors independently completed an 11-item quality scale for each article; scores were averaged to give a final measure of article quality.
We identified 20 articles examining interventions designed to improve health among people with low literacy. The most common outcome studied was health knowledge; fewer studies examined health behaviors, intermediate markers, or measures of disease prevalence or severity. The effectiveness of interventions appeared mixed. Limitations in research quality and heterogeneity in outcome measures make drawing firm conclusions about effective strategies difficult. Only 5 articles examined the interaction between literacy level and the effect of the intervention; they also found mixed results.
Several interventions have been developed to improve health for people with low literacy. Limitations in study design, interventions tested, and outcomes assessed make drawing conclusions about effectiveness difficult. Further research is required to understand better the types of interventions that are most effective and efficient for overcoming literacy-related barriers to good health.
Low literacy, defined as an inability to read, write, and use numbers effectively, is common and is associated with a wide range of adverse health outcomes.1–4 The association between low literacy and adverse health outcomes likely represents 2 related underlying processes. First, low literacy may have a direct, negative effect on health. We expect this effect to be particularly important for conditions that require substantial and complex self-care on the part of the patient, because of the barriers to accessing and using health information, particularly written information. Second, low literacy may be a marker for other conditions, such as poverty and lack of access to health care, that lead to poor health.
Over the past 10 years, researchers have developed and studied a variety of interventions to improve the health of patients with low literacy. Some have addressed direct literacy-related barriers primarily by testing interventions to make health education materials easier to understand. Others have focused on indirect barriers by providing more general supportive interventions. We are unaware of previous efforts to systematically identify and critically appraise studies of interventions that attempt to mitigate the effects of low literacy. In this systematic review, we identify, describe, and evaluate interventions to improve health outcomes for patients with low literacy and reduce disparities in health outcomes associated with low literacy.
Our review of interventions to improve health in patients with low literacy is part of a larger review commissioned by the Agency for Healthcare Research and Quality (AHRQ) that examines two main questions: the relationship between literacy and adverse health outcomes and interventions to improve outcomes for patients with low literacy. The full report, including a more detailed description of the methods and full evidence tables, is available at http://www.ahrq.gov.2 The systematic review of the studies that examine the relationship between literacy and a range of health outcomes is reported separately in the Journal of General Internal Medicine.3
We developed eligibility criteria in consultation with an advisory panel of experts in literacy-related research including physicians, health services researchers, nurses, and policy experts. To be included, studies had to 1) be conducted in a developed country (defined as United States, Canada, Western Europe, Japan, Australia, or New Zealand; we used this criterion to increase saliency for our U.S. target audience); 2) be published from 1980 to 2003; 3) be written in English (in order to select for research most applicable to U.S. populations and because of our lack of translation capabilities for non-English language articles); 4) use a controlled or uncontrolled experimental design; 5) study more than 10 subjects; 6) measure literacy directly among participants; and 7) measure the effect of an intervention on at least one health outcome. We defined eligible health outcomes to be:
The literature search procedures, including search terms used, are described elsewhere.2,3 In brief, we searched MEDLINE, Cumulative Index to Nursing and Allied Health (CINAHL), Educational Resources Information Center (ERIC), Public Affairs Information Service (PAIS), Industrial and Labor Relations Review (ILLR), PsycInfo, and Ageline from 1980 to 2003 to identify relevant articles. The starting date was chosen based on advice from the expert advisory panel that few or no studies meeting our eligibility criteria were published before 1980.
We used the following key words for our search: literacy, numeracy, WRAT, Wide Range achievement, rapid estimate of adult, TOFHLA, test of functional health, reading ability, and reading skill. To identify additional relevant literature, we reviewed the National Library of Medicine Current Bibliography in Medicine-Health Literacy5 and the Annotated Bibliographies at the Harvard School of Public Health Department of Health Literacy Studies (http://www.hsph.harvard.edu/healthliteracy/literature.html).6 We also solicited articles from experts and peer reviewers.
To determine whether studies met our eligibility criteria, we first examined the abstracts of articles identified by our literature searches. One reviewer initially evaluated abstracts for inclusion or exclusion, using an eligibility checklist. If the first abstractor concluded the article should be included in the review, we retained it for the next phase of the analysis; articles excluded by the first reviewer received a second review and we retained them if the second reviewer did not agree about exclusion.
For full article review, one reviewer read each article and decided whether it met inclusion criteria. A second reviewer rereviewed all articles. The four senior Evidence-Based Practice Center (EPC) staff decided as a group whether to accept any once-excluded articles and resolved disagreements through discussion. Articles that did not meet our eligibility criteria were assigned a reason for exclusion. A full table of the excluded articles and reasons for exclusion is available at http://www.ahrq.gov.
The first reviewer (MP, SS, or DAD) initially entered data from an article into the evidence table; the second reviewer (NB) checked and edited all table entries for accuracy and consistency. All disagreements concerning the information reported in the evidence tables were reconciled by the two abstractors, in consultation with other team members.
Two independent reviewers rated the quality of each included article using a structured 11-item form based on the domains and elements recommended in a recent systematic review by West et al.7 The 11 items were distilled into 8 domains to create a quality score.2 We graded each study according to the adequacy of study population, comparability of subjects, validity and reliability of the literacy measurement, maintenance of comparable groups, appropriateness of the outcome measurement, appropriateness of statistical analysis, and adequacy of control of confounding. These elements were recommended because they represented a comprehensive but feasible means of identifying key factors that affect the possibility of bias.
Each of the 8 domains received a rating of “good,”“fair,” or “poor.” We then created a composite rating that gave each item equal weight. Specifically, we converted ratings for each domain into numeric values in which 0=poor, 1=fair, and 2=good. We totaled each evaluator's score for each article and then averaged the results to produce an article quality score. Articles with a mean score less than 1.0 were considered poor quality; those 1.0 to 1.49 fair quality; and those greater than or equal to 1.5 good quality.
We reconciled ratings in which one rater provided a score for the item and the second said the item was not applicable. Although our rating scale is based on the best available evidence for this type of assessment, it should be interpreted with caution, as it has not been validated.
The full report underwent extensive external peer review prior to journal submission. A list of peer reviewers is available at http://www.ahrq.gov. AHRQ also reviewed the full report and this manuscript, made editing suggestions, and gave approval for submission for publication.
Our initial literature searches identified 3,015 articles, of which 2,331 were excluded on review of the abstracts. Among the 684 articles that underwent full review, we excluded 611 and retained 73 articles. The main reasons for exclusion of studies at the full article review phase were: no original data (48% of articles), no health outcome (34% of articles), and no measure of literacy (10% of articles). Among the 73 articles retained, 44 examined associations between literacy and health outcomes and are reported in a separate publication.3 Of the 29 articles on interventions, we excluded 9 articles because they did not measure literacy in their study population8–15 or were conducted in a developing country.16
The 20 studies were of 3 types: randomized controlled trials (n =9), nonrandomized controlled trials (in which subjects were assigned to intervention or control groups by the day or the week or some other nonrandom fashion; n =8), and uncontrolled, single-group trials (n =3). The number of participants enrolled ranged from 28 to 1,744; most studies had between 100 and 500 participants. All but 2 studies were conducted in the United States. Most interventions and outcome assessments were administered in single sessions. Interventions to improve dietary behavior26,28,30,31,33 and 1 other study36 delivered multisession interventions and/or followed participants longitudinally to assess changes in outcomes.
All 20 studies used previously validated instruments to measure the literacy of each trial participant. The most commonly used instrument was the Rapid Estimate of Adult Literacy in Medicine (REALM; n =11), followed by the Wide Range Achievement Test (WRAT)4; 5 studies used other instruments. No intervention study used the Test of Functional Health Literacy in Adults (TOFHLA), which has been commonly employed in studies of the relationship between literacy and health.3,37,38
Only 5 controlled trials stratified their results by literacy level. Each of these studies measured knowledge as its only outcome.18,22,23,26,27 In addition to literacy, most studies reported participants' mean age, ethnicity, and mean education levels. Information on participants' income level and health insurance status was available for a minority of studies. These data are available in the full evidence report.2
Included studies tested a wide range of interventions for improving health outcomes in patients with low literacy. Most interventions attempted to make health information more accessible to patients with limited literacy. Primary types of interventions were easy-to-read printed materials (n =4) videotapes or audiotapes (n =4); CD-ROM, computer program, or interactive videodisc (n =3); and in-person instruction, individually or in groups (n =9). Interventions were sometimes compared against standard information delivery or materials known to be more difficult to read; other studies compared their interventions against usual care or no intervention. Some studies compared different formats or styles of information with similar readability levels.
Included studies measured the following types of outcomes: health knowledge, health behaviors (e.g., smoking rates, dietary patterns, self-care), biochemical or other intermediate markers (e.g., cholesterol levels, weight, blood pressure), and use of health services (e.g., mammography rates). Knowledge outcomes were used most frequently. Only 1 study directly measured health outcomes that participants could feel and report on directly, in this case depression. Other than dietary behavior, few outcomes were examined in more than 1 or 2 studies, making outcome-specific comparisons and conclusions difficult.
Twelve studies examined health knowledge or comprehension of health materials as their main outcome of interest.17–24,26,27,33,34 The effects of interventions on health knowledge were mixed: some found increased knowledge, and others found no effect. Two studies examined the effect of literacy on postintervention knowledge but did not compare or report the overall effect of the intervention itself.21,34 Five controlled trials stratified the effect of the intervention by literacy status.18,22,23,26,27 They reached mixed conclusions about the relationship between literacy level and intervention effectiveness. In a controlled trial among patients at a sleep apnea clinic, Murphy et al. used a written 11-item questionnaire to compare the effect of a videotape educational tool against the effect of a brochure written at 12th grade level (similar to the grade level of the video script).26 No net effect on knowledge was observed for patients with high or low literacy.
Michielutte et al. performed a randomized trial to examine the effect of an illustrated brochure on cervical cancer compared with a brochure using bulleted text only. Readability levels were similar (8.4 grade level vs 7.7 grade level).23 Overall, there was no difference in knowledge on postintervention questionnaires. Patients with literacy scores below the median (46) on WRAT understood the illustrated materials better than the bulleted text version (61% vs 35%; P =.007). For patients with higher literacy, no difference was detected (70% vs 72%).
Wydra performed a randomized trial to examine the effect of an interactive videodisc to improve knowledge of self-care for cancer fatigue symptoms compared with no intervention among cancer patients.27 Patients who received the intervention had greater self-care knowledge, but this effect was not related to the literacy level of the patient as measured on WRAT (P =.31).
Davis et al. performed a controlled trial comparing a locally developed polio vaccine information pamphlet designed for patients with low literacy against a Centers for Disease Control and Prevention pamphlet that had also been designed for easy readability (both brochures were written at the sixth grade level).18 Patients with low literacy (third grade reading level or less) did not differ in their comprehension of the 2 pamphlets; among persons with higher literacy levels on REALM, the locally developed pamphlet was associated with better comprehension.
Meade et al. examined the effectiveness of educational materials on colorectal cancer screening that were intended to be appropriate for persons with low literacy in a randomized trial of 1,100 patients at the Milwaukee County Hospital primary care clinic.22 Participants were assigned to 1 of 2 interventions (a videotape or an easy-to-read brochure written at the fifth to sixth grade level) or to a usual care control group. Patients receiving either intervention had greater improvements in knowledge scores after the educational materials compared with the control group (26% for the video, 23% for the brochure, 3% for controls). Both low-and high-literacy groups (stratified at the seventh grade level based on WRAT) who received either intervention showed improved knowledge between the pre-and posttests compared with the control group. However, differences in improvement between the 2 literacy groups were small and not statistically significant.
Several studies examined the effect of interventions on health behaviors as their main outcomes.25,28,29,35 The behaviors studied included ability to perform breast examination, smoking, dietary patterns, or medication adherence. Outcomes were mixed.
Coleman et al. found that women given educational materials about breast examinations written at the third grade level and accompanied by photographs had greater knowledge and were more accurate in performing exams on silicone breast models than patients given materials at the same grade level with illustrations.35 Hussey found that seniors who were given verbal teaching concerning medication compliance improved adherence; adding a color-coded medication schedule did not provide additional benefit, however.29 Studies addressing dietary behaviors produced small or no changes in dietary behavior.26,28,30
Kumanyika et al. conducted a randomized trial to examine the effect of a special cardiovascular nutrition program for African Americans on lipid levels and blood pressure that included 4 monthly classes plus audiotapes.31 They enrolled 330 participants recruited from grocery stores in Washington, DC. They found no difference between groups in the change in cholesterol levels from preprogram to postprogram. Net difference in blood pressure was 3.2 mm Hg among women and 1.7 mm Hg among men, but neither result was statistically significant. Hartman et al. also found no significant difference in cholesterol levels with a nutritional intervention targeted to persons with low literacy.30
One study examined the effect of an intervention that included direct literacy skill building and measured the outcome of persistent depression. Poresky and Daniels conducted a randomized trial and found that a comprehensive family services center, compared with standard Head Start, could improve parental reading skill and reduce the prevalence of parental depression.36
We identified no studies of interventions that used a self-reported global health measure such as the SF-36 as the health outcome of interest.
One study examined an intervention to affect the use of preventive care services. In a nonrandomized controlled trial, Davis et al. found that an intervention consisting of a 12-minute video, coaching tool, verbal recommendation, and brochure improved mammography utilization by 11% at 6 months, but not at 24 months, when compared with a verbal recommendation and brochure alone.32
We did not identify any studies examining the effect of an intervention on costs, charges, or reimbursements.
We found no studies examining whether interventions for patients with low literacy affected health disparities based on race, ethnicity, culture, or age.
We rated 10 articles as having good quality, 9 as fair quality, and 1 as poor quality (see Table 1). Common and important limitations in design included 1) use of uncontrolled before-and-after designs; 2) failure to measure literacy or analyze results by literacy level; 3) failure to account for multiple comparisons in the analysis; 4) inability to isolate the impact of overcoming literacy barriers compared with other co-interventions, such as increased contact time; 5) failure to assign persons randomly, with concealed allocation, to the intervention and control arms; 6) incomplete statistical reporting; and 7) in some studies, an inadequate sample size to exclude clinically important effects.
Studies of interventions to improve the health of persons with low literacy have increased in number over the past 10 years but remain relatively uncommon. Most completed studies have examined the effect of interventions on health knowledge or behaviors; fewer studies have examined interventions designed to mitigate the effects of low literacy on intermediate markers, measures of disease incidence or prevalence, or use of health services. No research to date has examined how interventions affect the general health status of persons with low literacy or whether interventions can affect health care costs or health disparities based on race, ethnicity, culture, or age. Because too few studies examined each type of intervention (brochure, videotape, computerized tool, or oral presentation), we are also unable to comment about which types of interventions might be most effective.
Completed studies to date have found mixed results: some have shown positive effects on health, others have found no effect. The diverse range of outcomes examined limited our ability to draw conclusions about effectiveness. Differences in study quality, as measured by our rating scale, did not appear to explain differences in effectiveness. Although several studies showed improved overall outcomes, most had not been designed to measure whether the intervention helped the participants with low literacy less or more than (or equally to) patients with higher literacy. We identified only 5 studies that did measure whether an intervention had different effects in persons with low versus high literacy. These studies all used controlled designs, measured literacy in all participants, and stratified their results according to literacy level, but to date such studies have examined only knowledge outcomes. Their findings were also mixed with respect to differences in their ability to improve health knowledge based on the user's literacy level: some found that interventions worked similarly in low-and high-literacy patients, others found that low-literacy patients benefited more than high-literacy patients, and still others found the opposite.
Although our review is to our knowledge the first to systematically identify and evaluate interventions for low-literacy patients, it should be interpreted in the context of several limitations. First, as with all systematic reviews, its findings depend on the quality of the information in the published literature. Limitations of the available intervention studies include frequent use of nonrandomized designs and nonconcealment of allocation; infrequent reporting of how health outcomes were assessed, including whether assessors were blinded to literacy and intervention status and whether the questions were administered in ways that would allow accurate responses by participants with limited literacy; poor description of interventions; and use of multimodal interventions that make it difficult to know which portions produced positive effects. Moreover, published research has focused on knowledge rather than more meaningful health outcomes, and we encountered so many different outcomes that quantitative synthesis (meta-analysis) of results was not possible.
Our review process also had some limitations. We did not include studies that did not measure literacy directly. As such, we may have failed to identify interventions that could be beneficial for patients with low literacy. Our quality grading system, although based on previous research, has not been independently validated, relies solely on information reported in published articles, and should be interpreted cautiously. In addition, we did not, for time and resource constraint reasons, conduct dual, independent, blinded review of all articles for inclusion or for abstraction of information into evidence tables. We believe, however, our modified process allows sufficient rigor to minimize potential selection bias.
Because currently available intervention studies leave many important questions unanswered, additional research is key to advancing this field. In particular, we need more intervention studies that examine whether the association between low literacy and adverse health outcomes is mainly direct (meaning that outcomes could be improved by interventions designed to overcome limitations in reading and quantitative reasoning) or indirect (such that outcomes might be better addressed by focusing on other underlying causes of health disparities such as poverty, lack of access to care, or racism).
Future studies should carefully specify research questions and comparisons to allow stronger conclusions to be drawn about the interventions' true value and to help clarify whether interventions work directly by affecting the ability of patients to understand health information or indirectly by overcoming other barriers.
Intervention studies should stratify results by literacy level. Without such analysis, readers cannot determine whether the intervention is more or less effective in the low-literacy group and whether it helped to ameliorate the differences in outcome according to literacy status. We have highlighted in this report the studies that perform this type of analysis. All used health knowledge as the main outcome, but their conclusions about the interaction between the effectiveness of the intervention and patient literacy levels were mixed.
Intervention studies have generally focused mostly on short-term knowledge outcomes or health behaviors. Future studies should link short-term knowledge changes to important, longer-term health outcomes that matter more directly to patients and their families. Interventions aimed at changing health behaviors such as dietary intake for patients with low literacy also face the challenge that changing such behavior has been difficult for patients, regardless of literacy level. Further, many interventions had multiple components. Analyses that examine the individual effects of the key intervention components could significantly advance the field and help us determine “how much” intervention is enough to improve health.
Another limitation of the literature is that few studies presented details about the nature of the intervention materials beyond reading level or about how patient outcome assessments were performed. Research suggests that people with low literacy skills can learn appropriate self-care tasks if the tasks are organized and presented in a fashion that facilitates incorporation into everyday life, but doing so effectively requires more than just developing easy-to-read written materials. More studies that examine and describe the tasks required for effective self-care and measure the ability of patients to learn those tasks may lead to more effective interventions. For example, rather than asking diabetes knowledge questions, researchers could analyze patient logs of insulin use based on glucose levels.
Finally, we need to conduct more studies that examine the effect on health outcomes of teaching persons with low literacy skills to read better. To do so will require collaboration between health researchers and experts in adult education and literacy training.
This review has important implications for practitioners as well. Health care providers, including physicians, nurses, and other health care personnel, should be alert to the widespread problem of low literacy, and should consider how to convey important health care information in ways that do not require advanced reading skills. They should have access to tools that have been shown to be effective, including videotapes, computer programs, and group education curricula. Practitioners can use the “teach-back” method to check to see that patients understand health information, a technique that has been associated with better outcomes in an observational study of diabetes.39 Structural changes in the way care is delivered, such as the use of disease management programs, may also have important benefits for patients with low literacy.40–42
Supported by a contract to the Agency for Healthcare Research and Quality (290-02-0016), Rockville, MD.