The objective of this study was to develop a data-driven classification scheme for lay comprehension errors, derived on the basis of two representative medical documents, intended for patients’ reading under participatory health care model. Without being a comprehensive taxonomy, this scheme provides a starting point for the important task of categorizing and remediating such errors. From a practical perspective, this work provides a description and an insight into possible causes of several error types than can be remediated via a combination of educational and informatics approaches. From a theoretical perspective, we provide a proof of concept for methodological feasibility of the task. Future efforts at developing a comprehensive taxonomy may use the combination of document retelling and content analysis, but apply it to a broader range of document types and a greater number of documents. As our documents had little numerical data, we were not able to elucidate common numeracy-related errors, such as problems with interpreting graphs and risk values and dosage conversions. Participants in our study had above-average education level, were healthy, and did not have an intrinsic motivation to understand the two documents. We should also be cautious interpreting the results of a study where all participants were recruited at a single academic site. At the same time, because it included not only students and faculty, but also staff, our sample was broader than a typical university sample. Their healthy volunteers’ status and thus insufficient motivation could increase the number of comprehension errors, while the level of education had the potential of decreasing it. Future studies should draw upon a more diverse sample of participants, which would include more variability of demographic characteristics and health status.
Besides the narrow scope, limitations of our study involve some overlap between categories. For example, confusion between two similarly sounding terms, such as hypoglycemia and hyperglycemia--most likely caused by the lack of familiarity with professional medical terminology--was categorized as a terminology error. Statements that demonstrated lack of understanding of scientific concepts (e.g., functions of hormones and the role of insulin in sugar metabolism) were classified as errors in clinical concepts. Although incorrect usage of the terms or incorrect conceptual explanations often constitute
errors, they also often underlie
errors in statements about findings, diagnoses, and procedures. When a participant says that the patient had been diagnosed with “diabetic neuropathy,” this confusion of “nephropathy” and “neuropathy”, classified as an error in diagnosis, is likely to be caused by the lack of terminological knowledge. Similarly, insufficient biological knowledge and non-normative beliefs (or theories) of health and disease not only clearly underlie errors in clinical concepts, but may also be at the root of findings and diagnosis-related errors. This blurring of categorical borders in our scheme is related to the difficulty in distinguishing between the two aspects of errors, as described by JCAHO [40
], 1) things that go wrong and 2) reasons why this happens. The difficulty is methodological and relates to the nature of the task, that is, reading comprehension. In comprehension, causes of errors are largely cognitive, related to lapses in memory and attention and insufficient knowledge. However, the outcomes are also cognitive (e.g., accuracy of verbal answers). As a result, distinguishing between the cause (specific health beliefs and theories or lack of terminology knowledge) and the effect of a misunderstanding (e.g., incorrectly recalled diagnosis or misuse of a medical term in an explanation of a disease mechanism) is more difficult than in medical errors that involve actions (e.g., administering a wrong medication with a similarly-sounding name, due to terminology confusion). This also results in a scheme where categories differ in their “depth”, as some error classifications are likely to be related to lay beliefs and theories, while others (e.g., misspellings) may be more straightforward in their origins. A two-dimensional taxonomy, distinguishing between the causes of errors and the errors themselves, would address the issue of overlap, but developing it is challenging for the same reasons that cause the existing overlap.
The existing classification scheme has some insights for development of electronic medical documents. While PHRs and other online consumer health resources have the potential of improving patients’ and consumers’ experience with participatory healthcare, findings of this study suggest that lay people need support comprehending medical documents and, by extrapolation, authoring documents. Even in this group of highly educated participants, comprehension errors were frequent. They were also broad in scope, including understanding of research conventions, biomedical concepts, medical facts, and professional medical terms. Due to the largely narrative nature of our documents, we did not record many numeracy-related errors, but work by other authors suggests that problems with numeric conversions (e.g., dosages) and data representations are also common ([43
]). The diverse range of error types suggests that informatics support to document comprehension should be multi-faceted, and at the same time tailored to specific problems. At the present time, most tools are directed at translating professional medical terminology into consumer-friendly terms (e.g., Zeng-Treitler and colleagues [27
]). This appears to be necessary, but not sufficient: EHR/PHR and informed consent documents also need contextually relevant educational materials, easy to read summaries of findings and their interpretations, explanation of ranges and values of tests results, glossaries of medications’ names, and more.
While working on improving laypeoples’ experience with medical documents, it is also important for informaticians and educators to remember that making patients’ understanding of the documents mirror that of their healthcare providers is neither realistic, nor desirable. In the absence of specialized biomedical knowledge and clinical experience, laypeople will ascribe different level of importance to different statements, remember different facts, and organize information differently. The goal of lay comprehension support is not to position patients as professionals, but to enable them to work with professionals in the most effective way possible. We should also keep in mind that differences in lay and professional views on treatment and care should not be reduced to patient comprehension errors. Patients and healthcare professionals have different models of health and disease, which may produce somewhat discrepant value and belief systems. Physicians’ reasoning and decision-making is guided by the “disease model,” in which health problems are prominently connected to pathophysiological mechanisms. Patients, on the other hand, are guided by the “illness model,” which may include a combination of formal knowledge, naïve health concepts (“folk biology”), personal experience, and social and emotional implications of the disruption of normal routines caused by the illness (Patel, Arocha, & Kushniruk [68
]). While discussing patient-provider communication across these different belief systems is beyond the scope of this work, it is important to be aware that not all misunderstandings can be corrected via informatics support of “plain language” in medical documents.
This study classified participants’ comprehension errors into nine categories and twenty three subcategories; there are other possible error types that were not elucidated here because of the nature of the two documents. Not all error types are equally critical for comprehension and not all are likely to have similar impact on health behavior and decision making. Of the errors in this study, the one raising the most concern is interpreting the aim of the clinical trial. Although the purpose of the trial was phrased in the document as “to establish a new assessment method for glycogen metabolism,” it was frequently misconstrued as developing treatment. Twenty nine of the eighty participants made that error; another twelve participants misinterpreted the purpose of the trial without defining it as treatment. Participants in our study were healthy, well-educated individuals, who were reading the description of the trial in a comfortable emotional state, without the overwhelming anxiety that would accompany a bad diagnosis given to themselves or to a loved one. If these people had difficulty understanding the trial’s objectives, we should expect similar or greater difficulty in the general population of patients and caregivers. Misunderstanding objectives of clinical trials and routine procedures is likely to lead people to enroll into trials without true informed consent or seek inappropriate treatment. It may also lead to a targeted population missing information about trials of potential interest: if lay people do not understand the research purpose of trial descriptions, they are not likely to bring those trials to their physicians’ attention and ask whether the trials are right for them. While pre-empting this error is critical, it is also challenging without careful one-on-one involvement with a health professional. Attempts to explore informatics solutions may rely on cognitive science research of the impact of text signals (e.g., bullets, section headings, color highlights) on comprehension and attempt to use these signals to make research objectives more prominent. The problem of misunderstanding research objectives, however, cannot be corrected by information alone, as it is likely to be tied to the public’s lack of understanding of clinical research objectives and conventions. This study suggests the importance of discussing the rationale behind clinical research, benefits to society vs. direct benefits to participants, and different types of research studies (e.g., interventional vs. observational) in health and science education.
Another error category, common in this study, involved medication-related errors. Twenty-four of eighty participants made medication errors when reading the clinical trial document; fifty-seven of eighty made them when reading the visit note. Most errors had to do with medication names, particularly with misspelling and confusing brand names of medications. One of the limitations of this study is that it is impossible to ascertain whether the misspellings were, indeed, knowledge errors (participants did not know
how to spell the name) or attentional slips resulting in typographic errors. While we are not aware of research into consumers’ confusion of medication names, as opposed to confusion in pharmacies [69
] and medical centers, the extreme human impact of medication errors ensures that the phenomenon is well-studied in the medical literature that focuses on health professionals. For example, Koczmara and Hyland [70
] report on confusion of two particular drug names – Plavix and Pradax – and the implications for critical care nursing. Senger et al.[71
] write about drug misspellings as an information retrieval problem in Heidelberg University’s drug information system, describing error types as cognitive, phonetic and typographic, with typographic the most problematic. “Look-alike, sound-alike” (LASA) errors are defined by Basco et al. [72
] as “the erroneous prescription or delivery of a drug because the name of the drug (generic or brand) is similar in appearance to or sounds like another drug”. Not surprisingly, lay people have difficulty with aspects of medication names that are challenging to health professionals, and need help distinguishing among similarly-sounding medications and dealing with spelling of medication names. In most consumer interactions with medication names lay individuals have to state medication names orally or recognize their spelling by professionals, rather than spelling the names themselves. However, commonality of look-alike, sound-alike medication names suggests that the issue requires further investigation. It is tempting to speculate about the role played by DTC (direct to consumer) marketing of pharmaceuticals in the behavior of our participants. Regardless, these findings have implications for consumers’ information retrieval behavior and ability to self-educate using materials they find online. Consumers who have confused the Flovent they really need with the Flomax they half-remember will be unable to make much sense of information about either medication in any medium. Compared with understanding of research objectives and clinical concepts, supporting recall of medication names is within an easier reach for informatics. PHRs can include medication name spell-checkers, specify medication function, and provide names of similarly sounding medications and ask verification questions.
In discussing the theoretical issues around our classification scheme, we mentioned some overlap between two aspects of errors: the “what” and the “why” of things going wrong, or the underlying causes and the stated inaccuracies themselves. In our classification scheme statements that are clearly indicative of misunderstanding clinical concepts and not pertaining to findings, diagnoses, devices or procedures, make up a separate coding category. However, insufficient conceptual / biological knowledge or non-normative beliefs (theories) of health and disease are also likely to be the reason behind many errors related to misreporting findings and diagnoses. Knowledge of biological concepts is typically acquired over years of formal education, and is best addressed in the K-12 educational system. Research in science education suggests that solid biological knowledge, indeed, often underlies accurate health reasoning and effective health information seeking [73
] However, when point-of-care remediation is necessary, informatics can provide it via tailored and contextualized educational materials, exemplified by Baorto, Li and Cimino’s [26
In summary, this study suggests that lay people have difficulty reading medical documents, comprehension of which is essential for meaningful participation in their care. It also suggest that errors that people make can be classified into a manageable number of hierarchical categories, which are useful for thinking about ways to support lay users of electronic medical documents. Most common errors made by the participants in this study pertained to understanding conventions and objectives of clinical research, knowledge of health concepts and corresponding recall of medical findings and diagnosis, medical terminology and spelling, and problems with medication names. Future research will fine-tune these categories and identify new challenging areas, supporting tools for helping patients and consumers dealing medical documents.