The Reading Comprehension Section of Short Test of Functional Health Literacy in Adults (S-TOFHLA), one of the commonly-used and validated measures of health literacy, was used as a model for the Nutritional Literacy Scale (NLS) [12
]. The first version of the NLS was constructed from declarative sentences found in several nutritionally-related websites such as Mayo Clinic's Food and Nutrition Center, Tufts Nutrition Navigator and the USDA Center for Nutrition Policy and Promotion. Because the initial version of the NLS was used in a project on the metabolic syndrome, the items reflected cardiovascular-related topics such as "heart-healthy" eating, saturated fats, and portion size. Following the model of the S-TOFHLA, the modified Cloze procedure was used in which one or more words are removed from a sentence. Each sentence then includes several different options and the respondent picks the one that "fits" the best. For example, a sample item such as "Losing __________ can be a challenge" might be followed with choices such as (A) weight, (B) calories, (C) fiber, and (D) vitamins listed in a multiple-choice format. The Cloze procedure has been used for many years as one way to measure reading comprehension [13
The initial version of the NLS contained 21 modified-Cloze items in four-option multiple choice format and was pilot-tested on 132 adults, including family medicine patients, people taking courses at a local university, municipal employees and community members. The pilot subjects were able to complete the NLS without difficulty. However, a few of the items needed to be revised so as to better reflect comprehension, rather than knowledge of nutritional facts. A revised NLS containing 22 items was used in a second study with 103 adult patients in a family medicine practice different from the pilot site. To aid in the assessment of construct validity, the patients also completed the S-TOFHLA. Their medical charts were abstracted for information on cardiovascular variables such as the diagnoses of hypertension or diabetes. As evidence of construct validity, the two literacy measures – health and nutritional – were correlated (r = 0.69). Also, the patients with diabetes or hypertension had lower literacy scores than those without these diagnoses. The NLS also showed reasonable internal consistency by Cronbach's alpha coefficient (0.83).
Based on these data, a review of the item responses and to increase content validity, the scale was lengthened to 32 items. The scale was subsequently shortened to 28 items, following suggestions from a journal reviewer and an assessment of the statistical data for each item. Content areas such as organic foods, fiber, calcium and sugar were added to the original version. In general, items within each content area are ordered from the easiest to the more difficult, based on the pilot data. A total, number-right score is used for analysis. A page of demographic questions on age, gender, ethnicity and education is included. Because one goal in developing the scale was to be able to have patients complete it in the clinical office or by mail, the scale is untimed.
The 28-item version of the NLS was completed between 2004 and 2006 by 341 patients in four separate administrations. Three groups of patients were part of a University-based family medicine practice. One of these groups (Group 4) specifically included overweight and obese patients. Group 2 consisted of patients in an integrative medicine practice. They were included because they were known to be interested in nutritional issues, and because they were drawn from a less diverse population than were the family medicine patients. Because the groups were demographically diverse, combining them yielded a dataset that was potentially "richer" for analysis purposes. For three of the groups, patients were approached in the office by an assistant who requested their participation. While there were no explicit exclusion criteria for these groups, the assistants work in the office on numerous studies and are able to exclude patients in distress or those who are thought to be unable to complete the tasks based on their responses to the verbal consent script the assistants use. Although a random selection protocol in the office is impractical, the overall demographics of the practice were known and during the data collection phase, groups of patients such as younger men were targeted to try and keep the overall group representative. Patients in three of the groups also completed the Reading Comprehension Section of the S-TOFHLA, and they were given a phone card as an incentive. Because the providers in Group 2 did not want to place too much of a burden on their patients, they did not complete the S-TOFHLA. Lastly, the patients in Group 3 completed the scales by mail. Because of the potential introduction of "method variance," these data were originally not going to be included. However, as there was substantial overlap in the distribution of scores for this group and the others, they were included in the final analysis.
The item responses were entered into an Excel workbook and checked for accuracy. For the analysis, all the worksheets were imported into a statistical package (SAS Version 9.1 for PCs). The Pearson correlation between the two literacy measures was calculated as the estimate of construct validity of the NLS, with Cronbach's alpha as the estimate of internal-consistency. All data collection and consent procedures were approved by the University's Institutional Review Board.