To our knowledge, this is the first large multicenter study that validates brief screening questions for detecting inadequate health literacy. Using either the REALM or the S-TOFHLA as the gold standard, the question “Confident with Forms” performed significantly better for detecting patients with inadequate health literacy than the other 2 questions. For identification of the broader group of patients with either “inadequate or marginal” health literacy, the individual performance of each of the 3 health literacy screening questions was weaker.
This study had important strengths. We recruited a random sample of patients from multiple VA centers to obtain a more representative sample of the patient population and improve the generalizability of our findings. In addition, the large sample size allowed us to estimate the performance of these questions with much greater precision. Finally, this is the first study that compared the performance of the screening questions against 2 commonly used health literacy assessments, the REALM and the S-TOFHLA in the same sample.
The REALM and S-TOFHLA are the most widely used measures for health literacy. Each has been shown to predict knowledge, behaviors, and outcomes. Although these 2 tests are highly correlated,13
we discovered that the performances of the questions had a higher AUROC when the interview comparison standards were defined by the REALM compared to the S-TOFHLA. It is likely that the REALM and the S-TOFHLA measure different capacities.28
The REALM is a word recognition and pronunciation test that measures the domain of vocabulary. The S-TOFHLA measures reading fluency and consists of a reading comprehension section to measure prose literacy.
The optimal cut point of a screening test in a particular setting depends on several factors including test accuracy, prevalence of inadequate health literacy, costs of testing and false positive classification and the benefits of identifying true positives. If the objective of screening is to detect most persons who lack sufficient reading skills, we would want to choose a test cutoff with high sensitivity and low negative likelihood ratio so that persons who test negative are very likely to have adequate reading skills. However, if the objective of screening is to correctly identify those persons with low health literacy, one would choose a test cut cutoff with high specificity and high positive likelihood ratio so that persons who test positive are very likely to have inadequate health literacy. However, the implications of a positive or negative test vary dramatically depending on the prevalence of inadequate health literacy in the screened population. For example, a response of “Somewhat” will be the optimal screening threshold for Confident with Forms, in many settings. Using the sensitivity and specificity from the REALM (83% and 82%, respectively) and the prevalence of inadequate health literacy of 4.2% from our study sample, “Confident with Forms” would result in a posttest probability of inadequate health literacy of 21% at this cut point.29
If the same cut point was used for “Confident with Forms” in a setting with a higher prevalence of inadequate health literacy (35%) as reported in a previously published study,30
a positive screen would raise the posttest probability of inadequate health literacy to 76%.
Because the S-TOFHLA and REALM are not practical in busy clinical settings, a single question to screen for inadequate health literacy may be useful in clinical practice and research. The question “Confident with Forms” could be asked unobtrusively in busy clinical settings and may be less likely to induce anxiety or shame. Patients who screen positive for inadequate health literacy could be offered interventions using special methods of communication and assistance to allow them to successfully navigate the health care system. Finally, a single screening question could increase the feasibility of conducting needed research to develop effective interventions for patients with poor health literacy.
This study has several limitations. First, our study was conducted in a large population of VA primary care patients that may not be generalizable to other settings. Owing to the small proportion of women in our study, we were unable to determine whether the predictive abilities of these questions differed among women and men. However, our results were similar to those of Wallace21
who recruited patients from a university-based primary care clinic that suggests that the performance of these questions in other clinical settings may be similar. Second, our participation rate for this study was 41%. The comparison of respondents to non-respondents demonstrated the non-respondents were more likely to be older, have lower educational attainment, and have lower socioeconomic status, which suggests that non-respondents may have had lower health literacy than participants. This would have underestimated the prevalence of inadequate and marginal health literacy and could have resulted in a biased assessment of screening performance. Third, this study did not evaluate demographic characteristic such as age and education alone or in combination with the screening questions. These characteristics may be an alternative method to quickly identify patients with low health literacy.
In summary, we confirmed results from a previous study that the question “Confident with Forms” may be useful for detecting patients with inadequate health literacy in a VA population. Given the documented association of poor health literacy and health outcomes, these questions are an important advance toward being able to practically identify patients who might have difficulty understanding and acting on health care information.