|Home | About | Journals | Submit | Contact Us | Français|
Despite their significance, little is known about diagnostic errors in general pediatric practice other than data from malpractice claims. We surveyed pediatricians to elicit their perceptions about frequency, contributing factors and potential systems and provider-based solutions to address diagnostic errors.
Academic, community, and trainee pediatricians at three tertiary care institutions and affiliated 109 clinics (N=1,362) were invited to complete the survey anonymously via an Internet survey administration service between November 2008 and May 2009.
Overall response rate was 53% (N=726). Over half (54%) of respondents reported that they made a diagnostic error at least once or twice per month; this frequency was markedly higher (77%) among trainees. Almost half (45%) reported diagnostic errors that harmed patients at least once or twice per year. Failure to gather information through history, physical examination or chart-review was the most common reported process breakdown, whereas inadequate care coordination and teamwork was the most common system factor reported. Viral illnesses being diagnosed as a bacterial illness was the most commonly reported diagnostic error followed by misdiagnosis of medication side-effects, psychiatric disorders, and appendicitis. Physicians ranked access to electronic health records and close follow-up of patients as strategies most likely to be effective in preventing diagnostic errors.
Pediatricians reported making diagnostic errors relatively frequently and patient harm from these errors was not uncommon. Our survey provides new data about the types of diagnostic errors and their causes and lays the groundwork for a concerted, multifaceted approach to reduce these errors in children.
Errors in diagnosis constitute a sizable proportion of medical errors in the United States and are responsible for significant costs and harm.1–10 For instance, in the Utah-Colorado study diagnostic breakdowns were the most common type (36%) of ambulatory preventable adverse events that led to hospital admission.10 Although medical errors related to treatment (e.g., medications, surgery) have received much-needed recent attention,11 errors in the diagnostic process remain relatively understudied.12 Data about diagnostic errors from pediatric practice settings are especially limited. To date, knowledge of these errors is limited mostly to events that resulted in malpractice claims. Error in diagnosis is the most commonly identified reason in closed pediatric malpractice claims (32%), with the highest median indemnity payment and defense expenses.13 However, diagnostic errors that result in claims may only represent a small proportion of all diagnostic errors; given their low frequency and high severity, they may not be representative of all the types of diagnostic errors that occur in routine practice.2,14 Furthermore, underlying contributory factors for litigated diagnostic errors might differ systematically from errors not litigated.14 In short, malpractice claims studies have provided useful data to understand diagnostic errors but may not represent the entire spectrum of diagnostic errors.
Errors of diagnosis occur when diagnosis is unintentionally delayed (sufficient information was available earlier), wrong (another diagnosis was made before the correct one), or missed (no diagnosis was ever made), as judged by the eventual appreciation of more definitive information. 4 Although diagnostic errors may occur when symptoms and signs of a disease are atypical or absent, they are largely related to cognitive (e.g., faulty data gathering or clinical reasoning) and systems-related factors (e.g., issues with policies, processes and procedures, organizational issues).2,4,15–28 This understanding of diagnostic errors is largely based on non-pediatric settings and populations.
The diagnostic process in children is unlikely to be entirely analogous to adults. The most prevalent types of diseases and their associated diagnostic processes are different (e.g., certain infections are more common whereas cancer is more uncommon in children). Other unique factors in pediatrics include systems issues, patient/caregiver issues, training of physicians, and practice workflow, all of which are likely to influence diagnostic errors. In order to enhance our understanding about the relative prevalence of diagnostic errors, contributory factors, and potential preventive strategies, we designed an anonymous survey to study pediatricians’ experiences with diagnostic errors. We also sought data on specific disease conditions commonly associated with diagnostic errors in pediatrics.
To obtain diverse perspectives and solutions, we sampled three types of pediatricians: academic (general pediatricians and subspecialists), trainees (residents and fellows), and non-academic community-based pediatricians. Academic pediatricians and trainees belonged to the University of Texas Medical School, Baylor College of Medicine, and Cincinnati Children’s Medical Center. Although we invited participation from all trainees and general pediatricians at the three study sites, we randomly invited only 50% of pediatric subspecialists at Baylor College of Medicine and Cincinnati Children’s Medical Center due to the large number of subspecialists at those sites. Community pediatricians were selected from large practice groups affiliated with the teaching hospitals of the latter two institutions and represented 109 practice locations. Institutional Review Board approval was obtained at all three study sites.
We were unable to find an existing survey that addressed our study questions. Building on prior empirical and theoretical research on diagnostic errors, we developed a comprehensive survey following an exhaustive review of both adult and pediatric literature. A psychometrician guided the survey development process, which included item writing and refinement using subject matter expert input, clinician review, and pilot testing. Content of survey items was informed by published studies, input from experts in patient safety and diagnostic errors, and feedback from a sample of pediatricians from our study sites. The survey was developed for Web-based administration utilizing the resources of a user-friendly commercial Internet survey administration service (www.SurveyMonkey.com).
After initial refinements of survey items, we convened four focus groups of local pediatricians to review the survey for readability, clarity, and ease of completion in a Web-based environment. The survey was then pilot-tested with 8 general pediatricians. To understand the burden of diagnostic error in relation to other types of error, we enquired about frequency and potential to harm for errors related to medications, surgery and other clinical activities in pediatrics. In addition, the 23 item final survey (see Appendix) assessed physicians’ perceptions of 1) the most common process breakdowns associated with diagnostic errors (e.g. issues with history and examination, test ordering, performance or interpretation, and follow-up issues as gleaned from recent literature2,4,7,8,26,29–31); 2) the most common contributory factors that lead to diagnostic errors; 3) the most common types of medical conditions associated with diagnostic errors; and 4) the most useful strategies for prevention of diagnostic errors. Therefore, for many questions we used response scales with forced-choice ranking of top three response choices rather than Likert-style responses. The survey also assessed demographic information, self-reported frequency of diagnostic errors, attitudes toward discussing diagnostic errors, and previous training about diagnostic error. Estimated completion time was 15–20 minutes based on pilot testing.
We identified potential participants from medical staff offices at each institution and sent each physician an e-mail invitation to complete the survey, followed by up to two e-mail reminders. As an incentive to participate, we provided a web link to a $10 gift certificate in the initial invitation. Participants accessed and completed the items anonymously; only the study site and pediatrician category of each participant were identified.
Data were downloaded from the Internet survey administration service and analyzed using SAS version 9.2. We generated descriptive statistics about the nature and frequency of diagnostic errors in aggregate and then separately for academic pediatricians, community pediatricians, and trainees. Many items asked respondents to rank a first, second, and third choice from a list of possible outcomes. To calculate these rankings, we used weighted averages computed as follows: if a respondent ranked a particular choice first, that choice was given 3 points; 2 points were given for a second choice and 1 point for a third choice (all other choices were scored as 0). We then computed the average of all these values over all respondents. To assess for non-randomness of ranking, we used Friedman’s chi-squared test to test for significant differences among choices marked by respondents, i.e. whether a certain item was ranked above or below others after removal of variation among respondents. We tested the items in pairs with the significance level adjusted for multiple comparisons. We tested for differences in response frequencies between pediatrician type and between study sites using chi-square tests. We used linear regression to examine the effect of gender, race, practice site, type of pediatrician, training related to diagnostic errors in medical school and residency and years of clinical experience post-residency on the outcome “self-reported rate” of diagnostic errors. We used a scale consisting of seven possible frequencies; the seven values were treated as a continuous variable. The potential covariates were examined individually for inclusion in the final model.
Between November 2008 and May 2009, a total of 1,362 survey invitations were sent to physicians at the three study sites: 194 to the University of Texas at Houston, 569 to Baylor College of Medicine, and 599 to the University of Cincinnati. Invitees included 516 academic physicians, 430 trainees, and 416 community-based physicians. The overall response rate was 53%; groupwise rates were 52% for academic physicians, 39% for community-based physicians, and 67% for trainees. However, not all respondents completed the survey in its entirety; the overall survey completion rate among respondents was 86%, groupwise completion rates were 87%, 88%, and 85% respectively. Chi-squared tests showed that there were no significant differences between physician types or between sites in response rate.
Academic, community, and trainee groups did not differ in racial/ethnic distribution (Table 1). About a quarter (27%) of respondents had been in practice for more than 10 years, and two-thirds (67%) performed mostly clinical duties. Approximately half (48%) of respondents received training about diagnostic errors in medical school and a somewhat higher number (59%) in residency; however, this training was largely informal in nature. Majority (81%) of respondents reported discussing the diagnostic errors they had made either some, most, or all of the time with colleagues.
Table 2 lists the average frequency rankings of clinical activities believed by pediatricians to be associated with error and potential for harm. The Friedman test showed significant differences among the ranked activities (p<0.0001). Overall, pediatricians endorsed errors in medication-related activities such as prescribing and administering as most prone to error and associated with the greatest potential for harm. Figure 1 shows the distributions (by pediatrician category) of self-reported frequency of error (regardless of harm) and frequency of error with harm. Over half (54%) of respondents reported that they made a diagnostic error at least once or twice per month. However, the frequency of self-reported errors that may cause harm was lower. Only 4% reported errors that harmed the patient at least once or twice per month, 18% reported that such errors occurred at least once or twice per quarter, and 45% reported they occurred at least once or twice per year.
In the regression model, nonwhite pediatricians had significantly lower self-perceived error rate and trainees had significantly higher reported rates than academic and community pediatricians. Neither gender nor number of years of experience post-residency had a significant effect on the self-reported diagnostic error rate. Collectively, provider characteristics explained a relatively small proportion of the variance (R2 = 0.17) in self-reported frequency of diagnostic errors.
Diagnosis of viral illnesses as bacterial illnesses was the event most frequently associated with diagnostic error, followed by misdiagnosis of medication side-effects, psychiatric disorders, and appendicitis (see Table 3). The Friedman test showed significant differences among the ranked diseases (p<0.0001). Tables 4 and and55 display respondents’ rankings of process breakdowns and contributory factors perceived to be associated with diagnostic error. Among various types of process breakdowns associated with diagnostic error (Table 4), failure to gather available medical information had the highest ranking followed by care not sought in a timely manner by patient/caregiver. With regard to details of specific contributing factors (Table 5), inadequate care coordination, teamwork and/or communication across clinical settings or providers received the highest rating of all system-related factors. Of all cognitive factors, inadequate data gathering or work-up was ranked highest overall, though trainees ranked inadequate data assessment as a more frequent contributing factor than data gathering. Of miscellaneous factors that lead to diagnostic error, all groups assigned time/workload issues the highest average ranking. Again, the Friedman test showed significant differences among all the ranked variables (p<0.0001) in Table 4 and and5.5. We further explored the association of “care not sought in a timely manner” with “heath literacy” and found a strong positive correlation (p<0.0005).
We also inquired about specific heuristics (important cognitive shortcuts in the face of complex situations) and biases that could affect medical decision-making. Overall, the type of bias with the highest average frequency rating was being too focused on a diagnosis or treatment plan. Another bias with a relatively high frequency rating included being misled by a normal history, physical, laboratory or imaging study result.
Table 6 lists the ranking of selected solutions to reduce diagnostic errors. Of provider-based solutions, close follow-up of patients was ranked as most likely to be effective by all three subgroups of pediatricians, followed by improving teamwork, more time in clinical encounters, and empowering patients and families to be vigilant about the possibility of diagnostic errors. For systems-based solutions, access to electronic medical records which provide comprehensive clinical data was ranked highest again by all three groups of pediatricians, followed by availability of diagnostic decision support tools. Community-based physicians ranked increased access to and availability of consultants and experts second. The Friedman test showed significant differences among all the ranked solutions (p<0.0001).
We surveyed academic, community-based, and trainee pediatricians at three study sites about diagnostic errors and found several new insights that may help understand and prevent such errors in the future. Out of 6 types of medical errors, errors in diagnosis were ranked fourth in frequency and third in potential for harm. However, pediatricians reported they made diagnostic errors rather frequently; over half (54%) reported they made a diagnostic error at least once or twice per month. Diagnostic errors that led to harm were also not infrequent; almost half (45%) reported diagnostic errors that harmed patients at least once or twice per year. The most frequent diagnostic error was viral illnesses being diagnosed as bacterial illness, followed by misdiagnosis of medication side-effects, psychiatric disorders, and appendicitis. Failures in data gathering (history, examination, chart review) and care delays by patients/caregivers were reported to be the most frequent process breakdowns. Of various interventions for preventing diagnostic errors, pediatricians gave highest rankings to close follow-up of patients and access to electronic medical records.
To our knowledge, this was the first study to assess diagnostic errors in any setting through a comprehensive survey. Our study builds on the knowledge available from malpractice literature to enhance the understanding of diagnostic errors that affect children, serving as a guide to development of strategies to prevent these errors. According to our findings, pediatricians believe that errors in diagnosis occur most commonly in a set of conditions that differ markedly from those mentioned in pediatric malpractice literature. For example, in pediatric malpractice literature, diagnoses of meningitis, appendicitis, pneumonia and testicular torsion are cited to be most often missed. 32,33 However, we found that most pediatricians believed that viral illness misdiagnosed as bacterial was the most common diagnostic error, followed by misdiagnosis of medication side-effects and psychiatric disease. None of these overlap with the top diagnostic errors found in claims files. Errors of diagnosis of medication related side-effects are relatively unexplored in contrast to medication errors related to prescribing and administering. Coincidentally, a recent analysis of 583 physician-reported diagnostic errors in adults also revealed drug reactions or overdose as second most common type of diagnostic error. 34 Similarly, the concept of potential misdiagnosis of psychiatric disease in children although not entirely new,35 has not been discussed in context of malpractice claims and needs to be revisited with more specificity in future work. Another advantage of using this methodology is that pediatricians provided rich details about the most common process breakdowns and contributory factors, some of which are much harder to obtain from reviewing malpractice claims or medical records.36
We also solicited providers’ rankings of proposed strategies to prevent diagnostic errors. Despite ongoing debate about benefits of electronic health records,37 all groups of pediatricians ranked electronic records as the best system-based solution. Other strategies, such as diagnostic decision support tools and techniques to ensure timely follow-up of certain patients,29,38 may need to be prioritized in research on preventive strategies. Pediatricians also believed that an important factor leading to diagnostic error was patients’ and caregivers’ failure to seek care in a timely manner. Strategies to empower caregivers and patients should be studied and implemented in the context of patient safety research. Physicians often do not know the outcomes of the patients whose diagnosis they miss, hence feedback pathways to relay changes in diagnosis back to the original physicians are likely to be useful.29
Our findings provide valuable data to inform ongoing patient safety and medical error training efforts. The lack of emphasis on formal training for diagnostic error provides future opportunities to redesign the curriculum for both future pediatricians and practicing physicians.39,40 For instance, both academic pediatricians and trainees believed that failure to gather available medical information through history and physical examination and/or review of previous charts was a common breakdown. It may be valuable to refocus education on this much neglected aspect of training through innovative techniques that use standardized or virtual patients and simulation.40–42 Training should also focus on the interplay of both systems and cognitive factors, which has thus far been underemphasized in the literature. Critical thinking skills and strategies to reduce cognitive biases should be taught in the context of teamwork and system-related factors (such as time pressure) that may affect diagnostic performance.43–47 Carefully designed forums for open discussions of diagnostic errors are needed and are likely to be well received; many residents self-reported a high number of diagnostic errors and most physicians reported that they discussed their own diagnostic errors with colleagues at least some of the time.
Our study had several limitations. Because we relied entirely on self-reported error data rather than actual errors and could not determine which reported errors caused patient harm, our findings may need to be validated through other data sources. However, obtaining physicians’ (and other caregivers’) perceptions about errors is important and widely accepted as a fundamental approach to understanding and improving safety.48,49 The content and design of our survey was based largely on adult literature, although we incorporated a significant amount of feedback from pediatricians prior to administration. Pediatricians also may not know they missed a diagnosis, so errors may be underreported. While our response rate was perhaps not high enough, we believe this does not jeopardize the quality or generalizability of our findings. Literature has shown that response rates from physicians have been lower than other participants and response rates have declined over time.50,51 Published surveys of physicians have a mean response rate of only 54%, versus 68% from non-physicians.50 Lastly, we only surveyed pediatricians through e-mail, which may have limited participation by those lacking reliable e-mail access; however, the trainees and academic pediatricians in our sample were expected to use institutional e-mail accounts, which would limit this bias for this sample.
In summary, pediatricians reported making diagnostic errors relatively frequently, and they endorsed inadequate data gathering, poor care coordination, and patient/caregiver-related delays as prominent contributing factors. Improved follow-up of patients and access to electronic health records were perceived as the most promising potential interventions. In contrast to previous literature, our findings may be more generalizable to routine practice and provide concrete targets for future training and interventions to prevent diagnostic errors in children.
The study was supported by an NIH K23 career development award (K23CA125585) to Dr. Singh, the Houston VA HSR&D Center of Excellence (HFP90-020) and in part by the Fulbright & Jaworski Educational Award.
These sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs.
All authors had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.