|Home | About | Journals | Submit | Contact Us | Français|
Diagnostic errors are an understudied aspect of ambulatory patient safety.
To determine the types of diseases missed and the diagnostic process involved in the cases of confirmed diagnosis errors in primary care settings and to determine whether record reviews could shed light on potential contributory factors to inform future interventions.
We reviewed medical records of diagnostic errors detected at two sites through electronic health records-based triggers. Triggers were based on patterns of patients’ unexpected return visits after an initial primary care “index” visit.
A larger urban Veterans Affairs facility and a large integrated private health care system.
Our study focused on 190 unique instances of diagnostic errors detected in primary care visits between October 1, 2006, and September 30, 2007.
Through medical record reviews, we collected data on presenting symptoms at the index visit, types of diagnoses missed, process breakdowns, potential contributory factors, and potential for harm from errors.
In 190 cases, a total of 68 unique diagnoses were missed. Most missed diagnoses were common conditions in primary care, with pneumonia (6.7%), decompensated congestive heart failure (5.7%), acute renal failure (5.3%), cancer (primary) (5.3%), and urinary tract infection or pyelonephritis (4.8%) being most common. Process breakdowns most frequently involved the patient-practitioner clinical encounter (78.9%) but were also related to referrals (19.5%), patient-related factors (16.3%), follow-up and tracking of diagnostic information (14.7%), and performance and interpretation of diagnostic tests (13.6%). A total of 43.7% of cases involved more than one of these processes. Patient- practitioner encounter breakdowns were primarily related to problems with history-taking (56.3%), examination (47.4%), and/or ordering diagnostic tests for further work-up (57.4%). Most errors were associated with potential for moderate-to-severe harm.
Diagnostic errors identified in our study involved a large variety of common diseases and had significant potential for harm. Most errors were related to process breakdowns in the patient-practitioner clinical encounter. Preventive interventions should target common contributory factors across diagnoses, especially those that involve data gathering and synthesis in the patient- practitioner encounter.
Primary care practitioners manage a wide range of increasingly complex and severe conditions through one or more relatively brief encounters. Thus, it is not surprising that the primary care setting is vulnerable to medical errors.1–6 Diagnostic errors (missed, delayed, or wrong diagnoses)7 are of increasing concern in this setting.8–14 However, data on the most frequent misdiagnosed conditions are scarce, and little is known about which diagnostic processes are most vulnerable to breakdown. Most current data on diagnostic errors in primary care are derived from studies of malpractice claims or self-report surveys.10,15–17 These methods introduce significant biases that limit the generalizability of findings to routine clinical practice. A recent report by the American Medical Association recommends that efforts be made to “dramatically” strengthen the research base for outpatient safety, especially in the area of outpatient diagnostic errors.18,19 Understanding the circumstances around which these errors occur in typical practice is a necessary step toward generating preventive strategies.
In prior studies, we used a set of electronic health record (EHR)-based “triggers” (automated database queries) to identify primary care visits that were likely to be associated with diagnostic error.20 Our triggers were composed of algorithms to detect unusual patterns of care, namely unplanned hospitalizations, return visits, or emergency room visits within a short time after an initial primary care encounter. Physicians performed record reviews of triggered visits and non-triggered control visits to identify diagnostic errors. Our primary objectives in the present study were to determine the types of diseases missed and the diagnostic processes involved in cases of confirmed diagnostic errors in primary care settings. This exploration could advance knowledge about conditions that are vulnerable to being missed in primary care and help prioritize error prevention strategies. Our secondary objective was to determine if record reviews could shed light on potential contributory factors to inform future interventions.
Our study focused on 190 unique instances of diagnostic errors detected in primary care visits between October 1, 2006 and September 30, 2007 in two large health systems. Both sites had integrated and well established EHRs, and large clinic networks through which they provided longitudinal care. Both sites provided care to ethnically and socioeconomically diverse patients from rural and urban areas.
Site A was a large urban VA facility with about 35 full-time primary care practitioners (PCPs), including physicians, physician assistants, and nurse practitioners, providing comprehensive care to approximately 50,000 patients. Most PCPs were physicians, some of whom supervised residents. Primary care encounters included both scheduled follow-up visits and “drop-in” unscheduled visits.
Site B was a large integrated private health care system with 34 family medicine primary care physicians who provided primary and urgent care to nearly 50,000 patients in 4 community-based clinics. Over half of the PCPs supervised residents.
Details about diagnostic error detection techniques used in this study have been published previously.20 Briefly, our trigger queries were: 1) a primary care index visit followed by an unplanned hospitalization within 14 days, and 2) a primary care index visit followed by ≥ 1 primary care/emergency room/urgent care visit(s) within 14 days. Trained physicians then reviewed all “triggered” records for evidence of diagnostic error. Reviewers were fellows from medicine subspecialty training programs or chief residents in medicine and were selected based on recommendations from faculty and interviews by our research team. They were instructed to judge diagnostic performance based only on data already available or easily available to the index visit practitioner to either make or pursue the correct diagnosis. Within these constraints, reviewers evaluated several aspects of EHR documentation (notes, tests, referrals, case evolution over time, etc.) to ascertain presence of diagnostic error. An error was judged to have occurred if adequate data to suggest the final, correct diagnosis were already present at the index visit, or if documented abnormal findings at the index visit should have prompted additional evaluation that would have revealed the correct, ultimate diagnosis. Thus, errors occurred only when missed opportunities to make an earlier diagnosis occurred based on retrospective review.21–23 In diagnostic error cases, reviewers recorded the disease condition that was missed. A sample of randomly selected control visits (i.e., visits that did not meet either trigger criterion) were reviewed for errors using the same procedure. In 212,165 visits at both sites, we found 190 diagnostic errors; 141 of 674 Trigger 1 records (20.9%), 36 of 669 Trigger 2 records (5.4%) and 13 of 614 controls (2.1%) contained diagnostic errors. An independent, second reviewer confirmed each error case. Elsewhere we have described additional details regarding our methodology for determining errors.20
For each confirmed case of diagnostic error, one reviewer performed an additional level of review to determine process breakdowns, contributory factors involved, and potential for harm. We designed a data collection instrument after reviewing the previous literature on diagnostic errors, which included studies based on malpractice claims,10 physician surveys,24,25 and medical record reviews.13,21,22,26–28 We categorized cases using a five dimension model of ambulatory diagnostic processes13 to indicate the point in the diagnostic process at which errors occurred: patient- practitioner clinical encounter, performance and/or interpretation of diagnostic tests, follow-up and tracking of diagnostic information, subspecialty and referral-related, and patient-specific processes. To assess the reliability of judgments related to process breakdowns, a second reviewer independently evaluated 10% of the error records; inter-rater reliability was quantified by computing Cohen’s kappa. Process breakdowns in the patient- practitioner clinical encounter could involve problems with history-taking, physical examination, ordering of diagnostic tests for further work-up, and review of previous documentation. Additional contributing factors were collected for the remaining four dimensions using a comprehensive list of factors based on previous literature.10,13,21,22,26–29 We used an 8-point scale to collect data on each error’s potential for harm (1 = no harm/no inconvenience; 8 = immediate or inevitable death).30 Finally, we collected data on the patient’s age, race/ethnicity, and gender, and the type of PCP at the index visit (attending physician, trainee, physician assistant, or nurse practitioner).
We generated descriptive statistics to quantify the frequency of clinical conditions associated with diagnostic errors as well as process breakdowns, contributory factors, and harm. We also compared patient and practitioner variables between primary care visits with and without diagnostic errors. We performed a t-test to compare the mean age of patients involved in visits with and without diagnostic errors. We compared all other patient characteristics as proportions, using Fisher’s exact test for categorical variables when the assumptions for the chi-square test were not met (two-tailed).
Table 1 summarizes the characteristics of patients involved in visits with and without diagnostic errors at each site. At both sites, the mean age of patients involved in visits with diagnostic errors was slightly older than in cases not involving errors (66.5 years vs. 62.7 years; P = .002 at Site A; 53.8 years vs. 45.6 years; P = .003 at Site B). Diagnostic errors did not seem to be significantly associated with the type of practitioner at either site.
We found 68 unique diagnoses that were missed (both the triggered and control groups). When cases from both sites were combined, pneumonia (6.7%), decompensated congested heart failure (5.7%), acute renal failure (5.3%), cancer (primary) (5.3%), and urinary tract infection or pyelonephritis (4.8%) were the most commonly missed diagnoses in primary care. However, the most frequently missed or delayed diagnoses differed by site (Table 2). In some cases, more than one diagnosis was missed. In most cases (85.8%), a different practitioner, either from the same specialty (43.1%) or a different specialty (42.7%), saw the patient on the return visit. In more than half of the cases (51.6%), errors were discovered because of the failure of the original symptom or sign to resolve. Other errors were detected upon evolution of original symptoms and signs (34.8%) or the development of new symptoms or signs (22.6%). Only approximately one-fifth of the errors were discovered as part of planned follow-up, such as when practitioner asked the patient to return within a certain time period (usually few days) for re-evaluation. In 96% of triggered error cases, there was a clear relationship between the patient’s admission or second outpatient visit and the presentation on index visit.
Table 3 lists the chief presenting symptoms, as documented by the index practitioner, that were present in 2 or more cases of diagnostic error. Of these, cough (sometimes associated with additional presenting symptoms) was the most common. Notably, in 22 cases patients did not have any specific chief presenting symptoms; this occurred in instances such as when established patients were following up on chronic medical issues or when new patients were visiting to establish care within the system. The chief presenting symptoms was directly related to the missed diagnosis in approximately two-third of cases (67.4%) (data not shown in table). However, documentation of adequate exploration and investigation of the chief presenting symptoms was lacking in 93 cases (48.9%).
Breakdowns were found to occur in all 5 dimensions of the diagnostic process, and in 43.7% of cases more than one dimension was involved. The inter-rater reliability of process breakdown ratings was 0.56 (95% CI, 0.38–0.74). Most commonly, breakdowns occurred during the patient- practitioner clinical encounter (84.2%) and this finding was consistent across both sites (82.1% vs. 88.5% for Sites A and B, respectively). Breakdowns involving the patient- practitioner clinical encounter were most often judged to be due to data gathering and synthesis problems (i.e., cognitive errors) related to the medical history (56.3%), physical examination (47.4%), ordering of diagnostic tests for further work-up (57.4%), and failure to review previous documentation (15.3%). Two additional documentation-related problems were notable. First, no differential diagnosis was documented at the index visit in 81.1% of cases. Second, practitioner copied and pasted previous progress notes into the index visit note in 7.4% of cases; of these cases, copying and pasting mistakes were determined to contribute to more than one-third (35.7%) of errors.
Outside of the patient- practitioner encounter, process breakdowns also occurred in the areas of referrals, patient actions or inaction, follow-up and tracking of diagnostic information, and performance and interpretation of diagnostic tests (Table 4). The most common referral-related breakdowns were problems initiating a needed referral; in 1 of 10 error cases an appropriate expert was not contacted when indicated. Contributory factors for these processes are summarized in Table 4. No one factor was attributed to most errors in these categories, and no single factor in these categories contribute to 10% or more of all error cases.
The potential severity of injury associated with the delayed or missed diagnosis was classified as moderate to severe in 86.8% of cases (ratings 4–8 on the 8-point scale), with a mode of 5 (considerable harm) (Table 5). When we further broke down cases according to which diagnostic process was implicated in the error, we found a modal severity rating of 5 across all 5 processes.
We analyzed 190 primary care diagnostic errors, most of which were detected through electronic triggers. These errors involved a large variety of conditions that are seen commonly in primary care. Most diagnostic errors had potential for moderate to severe harm. Presenting symptoms for these conditions were highly variable and sometimes did not bear any obvious direct relationship to the condition that was missed. Most errors involved breakdowns of processes related to the patient- practitioner clinical encounter. Lower, though still meaningful, proportions of breakdowns occurred in the domains of referrals, patient factors, test ordering and interpretation, and follow-up and tracking of diagnostic information.
Our study is among the largest to address diagnostic error in routine outpatient practice and among the first to empirically evaluate the types of diagnostic errors that occur in primary care. Misdiagnosis of cancer has been considered among the most common diagnostic errors in primary care, mostly because of overrepresentation in studies of malpractice claims.10,11,27 However, a previous systematic review on diagnostic error in primary care found a wide range of conditions that were commonly misdiagnosed, including not only cancer but also myocardial infarction, meningitis, dementia, iron deficiency anemia, asthma, tremor in the elderly, and HIV.31 Most of the errors identified in our study involved missed diagnosis of a large variety of common conditions, as opposed to either a few selected conditions or rare or unusual diseases. Pneumonia and decompensated heart failure were most commonly missed, though they accounted for less than 13% of all errors. Furthermore, there were marked differences in the most common missed diagnoses across the two sites, largely because their local contexts and patient and practitioner populations were markedly different. For instance, practitionersat Site A were predominantly internists who cared for older veterans (who generally have more comorbidities), whereas at Site B, family practitioners cared for an overall younger population. However, at both sites, the most common process breakdowns arose within the patient- practitioner encounter. Because diagnostic errors in primary care involve a large number of heterogeneous conditions, future error reduction stategies should account for their common contributory factors and not just attempt to augment knowledge or clinical skills related to specific diseases, as such interventions may not generalize across diseases or care settings.
We were also able to report patients’ chief presenting symptoms associated with the commonly missed diagnoses, a topic that has been essentially unexplored.16 Knowledge about commonly implicated or “high risk” chief presenting symptoms could potentially lead to targeted interventions to decrease the likelihood of error, although many found in our cohort are fairly common in outpatient practice. Given the myriad symptoms that PCPs encounter in their daily practices, focusing on specific presentations is unlikely to form the sole basis for preventive strategies. Moreover, in approximately one-third of cases patients presented with symptoms that appeared to be unrelated to the missed diagnosis, which could easily divert the practitioner’s attention during the short span of the primary care visit. Another notable finding was the absence of documentation of differential diagnosis, a fundamental step in the diagnostic reasoning process. How patients present their chief symptoms to the PCP, and how this and other factors influence a PCP’s clinical reasoning within the context of a busy clinic, are areas ripe for further investigation.17,23
Our findings highlight the need to focus on basic clinical skills and related cognitive processes (e.g., data gathering within the medical history and physical examination and synthesis of data) in the age of increasing reliance on technology and team-based care to improve the health care system.32–34 Most process breakdowns were related to the clinical encounter, wherein practitioners are almost always pressed for time to make decisions.15 With the current emphasis on patient-centered medical homes that facilitate team-based care, patients might be able to access or interact with their practitioner more effectively. However, these new models of care might not produce the level of cognitive support needed for gathering and/or interpreting a patient’s key signs and symptoms effectively and safely. Furthermore, current forms of technology, including EHRs, are inadequately positioned to meet the needs of complex decision-making.35 Newer models of understanding how best to leverage both technological and non-technological strategies to improve practitioner and team situational awareness are needed. Additionally, more research is needed to determine why practitioners may not adequately search for data or synthesize findings, and how to best improve clinical skills and cognitive processes in the complex primary care environment. Although the current literature highlights isolated cognitive difficulties among practitioners (e.g., biases), and various interventions have been suggested to improve diagnostic decision-making (e.g., the use of checklists36 or second opinions), few cognitive obstacles have been sufficiently examined in the complex “real-world” primary care environment, and few interventions have been satisfactorily tested.37 Using the lens of missed opportunities in care rather than errors, institutions could create a new focus on discovering, learning from and reducing diagnostic errors.38 Our methodology can be one way to “proactively” discover such missed opportunities in real practice.20 Patient empowerment and engagement in the diagnostic process could add greatly to these strategies.
Our study has several limitations. Our methods may not apply to primary care practices that are not part of integrated health care systems. Moreover, ours was a retrospective study so hindsight bias remains a concern. Because of our study design, we did not have the additional benefit of practitioner debriefing, which if performed soon after error discovery, could provide additional useful data. The triggers we used are more likely to select for misdiagnosis of acute conditions and exacerbations of chronic conditions. These are important subsets of diagnostic errors39 but are not inclusive of all diagnostic errors; for instance, the errors discovered through our methodology were likely to have underrepresented diagnoses of conditions such as cancer that are less likely to emerge during an urgent presentation within a short time frame. We were able to achieve only moderate inter-rater reliability in determinations of process breakdowns, though it was similar to that reported in the landmark study of diagnostic error process breakdowns by Schiff et al.’s15 Finally, errors in our study were likely rated as more harmful because most of them were detected in the context of an unexpected hospitalization or return visit and this may not generalize to the universe of diagnostic errors in primary care. Nevertheless, the field of diagnostic error is fairly nascent and it is important to focus on any types of errors that cause harm or create the need for further care even though they might not be representative of all errors.
In conclusion, diagnostic errors in primary care include a heterogeneous group of common conditions, and most have potential to lead to moderate to severe harm. Most errors were related to patient- practitioner clinical encounter-related processes such as taking medical histories, performing a physical examinations, and ordering tests. Given the range of conditions associated with diagnostic errors in this setting, disease-specific efforts to reduce these types of diagnostice errors are unlikely to be sufficient. Thus, preventive interventions must focus on common contributory factors, particularly those that influence the effectiveness of data gathering and synthesis in the patient- practitioner encounter.
The views expressed in this article are those of the authors and do not necessarily represent the views of the Department of Veterans Affairs or any other funding agency.