|Home | About | Journals | Submit | Contact Us | Français|
Rationale: Treatment and prognoses of diffuse parenchymal lung diseases (DPLDs) varies by diagnosis. Obtaining a uniform diagnosis among observers is difficult.
Objectives: Evaluate diagnostic agreement between academic and community-based physicians for patients with DPLDs, and determine if an interactive approach between clinicians, radiologists, and pathologists improved diagnostic agreement in community and academic centers.
Methods: Retrospective review of 39 patients with DPLD. A total of 19 participants reviewed cases at 2 community locations and 1 academic location. Information from the history, physical examination, pulmonary function testing, high-resolution computed tomography, and surgical lung biopsy was collected. Data were presented in the same sequential fashion to three groups of physicians on separate days.
Measurements and Main Results: Each observer's diagnosis was coded into one of eight categories. A κ statistic allowing for multiple raters was used to assess agreement in diagnosis. Interactions between clinicians, radiologists, and pathologists improved interobserver agreement at both community and academic sites; however, final agreement was better within academic centers (κ = 0.55–0.71) than within community centers (κ = 0.32–0.44). Clinically significant disagreement was present between academic and community-based physicians (κ = 0.11–0.56). Community physicians were more likely to assign a final diagnosis of idiopathic pulmonary fibrosis compared with academic physicians.
Conclusions: Significant disagreement exists in the diagnosis of DPLD between physicians based in communities compared with those in academic centers. Wherever possible, patients should be referred to centers with expertise in diffuse parenchymal lung disorders to help clarify the diagnosis and provide suggestions regarding treatment options.
The treatment and prognosis of idiopathic interstitial pneumonias varies by diagnosis. An interactive clinical–radiographic–pathologic approach to diagnosis improves final diagnostic agreement.
Significant disagreement exists in the diagnosis of idiopathic interstitial pneumonia between physicians based in community and those in academic centers. Patients with suspected diffuse parenchymal lung disorders should be referred to centers with expertise in this area to help clarify the diagnosis and for suggestions regarding treatment.
Histopathologic subsets of idiopathic interstitial pneumonia (IIP) exhibit different prognoses (1–9). Therefore, an accurate diagnosis is critical to the management of patients with IIP. Clinical features, high-resolution computed tomography (HRCT) (10–14), and surgical lung biopsy (SLB) (15) all play a role in establishing a diagnosis. The American Thoracic Society/European Respiratory Society has recommended a dynamic, diagnostic, integrated process in which clinicians, radiologists, and pathologists exchange information in the determination of a diagnosis (16). We recently documented that such an approach among experts leads to changes in the final diagnosis compared with individual observers acting in isolation (17). In this study, we evaluated the agreement in classification of patients with suspected IIP in community and academic settings. As secondary goals, we examined the influence of an iterative diagnostic approach on diagnostic agreement in a community compared with an academic setting, and addressed features that influenced diagnostic approaches.
Data from patients referred to the University of Michigan Specialized Center of Research in the Pathobiology of Fibrotic Lung Disease between August 2002 and December 2003 were used for this study. Patients with suspected IIP were referred to the study center by participants in the University of Michigan Fibrotic Lung Disease Network (see section before References). Through the course of evaluation, all patients underwent a history, physical examination, complete pulmonary function testing, HRCT, and SLB. Patients without an HRCT scan or an SLB were excluded.
A standard form was used to collect clinical information, including symptoms, environmental exposures, comorbid illnesses, medications, smoking history, family history, physical exam findings, and serologic data. Pulmonary function data (spirometry, lung volumes, and diffusion capacity for carbon monoxide) and HRCT within 6 months of SLB were reviewed. Data from bronchoscopy (transbronchial biopsy and/or bronchoalveolar lavage) were only available in a minority of patients and were, therefore, not included in the data presented.
Case information was provided to three groups (community 1, community 2, and the University of Michigan) on separate days. Participants at the University of Michigan were expert clinicians, radiologists, and pathologists from five centers (within and outside the United States). On average, participants at the University of Michigan had been in practice longer and spend a greater amount of time in the evaluation and treatment of patients with interstitial lung disease (see Table E1 in the online supplement). The cases were presented with the same information and in the same order at each institution. We provided participants incremental information through five stages (Figure 1), as previously described (17). Briefly, clinicians and radiologists independently reviewed clinical information, followed by HRCT, and then discussed, as a group, the clinical and HRCT features (stages 1–3). As this was occurring, the pathologists were independently reviewing SLB specimens and assigning an independent histopathologic diagnosis. The fourth step included a group (clinicians, radiologists, and pathologists) discussion of the findings. During step 5, an attempt was made to reach a consensus diagnosis.
Each observer's diagnosis was coded into one of eight categories: idiopathic pulmonary fibrosis (IPF), nonspecific interstitial pneumonia (NSIP), bronchiolar/airway, hypersensitivity pneumonitis (HP), respiratory bronchiolitis interstitial lung disease/desquamative interstitial pneumonia (RBILD/DIP), cryptogenic organizing pneumonia (COP), interstitial lung disease with suspected underlying collagen vascular disease (ILD/CVD), and “other.” McNemar tests were subsequently used to test whether two probabilities of agreement conducted during different steps or by different raters were equal. A κ statistic allowing for multiple raters was also used to assess agreement in diagnosis. κ Scores are rated as almost perfect agreement (above 0.8), substantial agreement (scores between 0.6 and 0.8), moderate agreement (scores between 0.4 and 0.6), fair agreement (scores between 0.2 and 0.4), slight agreement (scores between 0.0 and 0.2), and poor agreement (scores below 0.0) (18). An estimating equation approach to the analysis of correlated κ statistics was used in comparisons of κ statistics estimated throughout the study and in producing confidence intervals for the κ statistics (19). SAS (SAS Institute, Cary, NC) macros, developed and described by Gwet (20), were used to obtain the numerical characteristics of the κ statistics.
A total of 39 cases were evaluated by the community and academic specialists. Clinically significant differences in diagnoses were present among the study participants.
Academic physicians displayed better agreement compared with community physicians (Table 1; Figure 2). The academic clinicians exhibited a very good agreement upon a step 5 final diagnosis (κ = 0.71), as compared with the community clinicians (κ = 0.44). The same was true for all previous diagnosis steps. This improved agreement occurred despite being more numerous than their community counterparts (n = 6 vs. n = 3) and, hence, less likely to reach agreement all else equal. κ Scores improved for both the community clinicians and academic clinicians as more information was provided (steps 1–5), although this was less impressive among the community participants (Table 1). The fias fias fias fias final diagnosis agreement between academic and community clinicians varied from 0.20 to 0.56 (Table 2)
There was greater interobserver agreement among the academic radiologists than among the community radiologists. κ Scores failed to improve for both the academic radiologists and community radiologists as more information was provided (Table 1; Figure 2). The final diagnosis agreement between the academic and the community radiologists was low (range, 0–0.34 [Table 2]).
There was greater interobserver agreement among the academic pathologists than among the community pathologists. κ Scores for academic pathologists were similar at all stages of evaluation, whereas community pathologists displayed improvement in agreement after discussing the case with clinicians and radiologists (Table 1; Figure 2). The final diagnosis agreement between the academic and community pathologists was low (range, 0.12–0.48 [Table 2]).
In general, κ scores for the clinicians appeared to be somewhat lower between the first and second stages than between the second and third stages, suggesting that clinical information altered HRCT interpretation more than did interaction with the radiologists (Figure 3). In addition, the κ scores appeared lower between the third and fourth stages than between the second and third stages, confirming the influence of pathologic interpretation on changing diagnoses.
Among radiologists, the provision of clinical information at stage 2 appeared to have a greater effect on academic radiologists, as suggested by lower intraobserver agreement among these participants between the first and second stages (Figure 3) compared with community radiologists. Interaction with clinicians (between second and third stages) led to few changes. The provision of pathologic information led to the greatest changes (stages 3 to 4). Slight changes were seen in intraobserver agreement among the participating pathologists with the provision of clinical and radiologic information (Figure 3).
Total or near-total (all agree except one or two observers) agreement was achieved in a minority of cases (Table 3). Most agreement occurred with a diagnosis of IPF for both community and academic physicians. Community-based physicians were more likely to make a diagnosis of IPF than were academic-based physicians.
We subsequently explored differences in diagnosis at a case-by-case level. Cases were grouped (Figure 4) by final diagnosis into cases in which the majority of observers felt the diagnosis was IPF, a disagreement between IPF and HP, HP, CVD, NSIP, disagreement between IPF and NSIP, bronchiolar/RBILD, COP, and “other.” For cases in which a split diagnostic opinion was present, there was a trend for community physicians to make a diagnosis of IPF and for academic physicians to assign a non-IPF diagnosis.
In 13 cases, the final majority diagnosis was IPF. In these cases, academic clinicians and radiologists were more likely to consider a diagnosis of NSIP or HP before the pathologic information compared with community physicians. Interestingly, there was near-complete agreement among community and academic pathologists at each stage. The fact that two cases (351 and 357) had a history of bird exposure did not seem to impact the diagnosis of IPF.
In three cases, there was a split between a diagnosis of IPF versus HP. Academic physicians seemed to favor a diagnosis of HP, and community physicians seemed to favor a diagnosis of IPF. Two of these three cases had a history of bird exposure. The listing of granulomas as a feature by pathologists (data not shown) varied within both academic and community pathologists, suggesting that HRCT appearance played a role in the final diagnosis as academic radiologists, clinicians, and pathologists were more likely to assign a final diagnosis of HP than were their community counterparts.
In four cases, the majority consensus diagnosis was HP. In these cases, the diagnosis appeared to be driven by the finding of granulomas by pathologists, as the prepathology diagnoses by clinicians and radiologists were extremely varied and only one case had a history of bird exposure.
In six cases, observers raised the possibility of an underlying CVD contributing to the pulmonary findings. Clinical information for these patients often included a known history of CVD or positive serologies.
In two cases, the final majority diagnosis was NSIP, although several observers selected IPF or HP as their first-choice final diagnosis. In four cases, there was a split in final diagnosis between IPF and NSIP. Academic clinicians and radiologists were more likely to assign a diagnosis of NSIP, whereas their community counterparts were more likely to assign a diagnosis of IPF. In general, both community and academic pathologists favored a diagnosis of IPF. These cases highlight the difficulty in making a “consensus” diagnosis of NSIP, especially in a community setting.
In the remaining cases, there were two cases of COP, one case each of bronchiolar disease and RBILD, and three cases with near-complete diagnostic disagreement.
Diffuse parenchymal lung diseases (DPLDs) are a diverse group of disorders with varied prognoses and response to therapy. Assigning a specific DPLD diagnosis to an individual patient is difficult and, at times, imprecise. However, making an accurate diagnosis, and, importantly, having a uniform diagnostic approach applied to patients wherever they are seen, is critical to the study and application of clinical trial data to individual patients. We previously demonstrated that an integrated diagnostic approach involving expert clinicians, radiologists, and pathologists results in an altered diagnosis compared with that of individual physicians working in isolation (17). In the current study, we expand these findings by examining the diagnostic agreement between community- and academic-based physicians using a dynamic interactive process involving pulmonary clinicians, radiologists, and pathologists. We demonstrate that: (1) clinically significant disagreement exists regarding the diagnosis of IIP among academic-based clinicians and between community- and academic-based physicians, with community physicians more likely to make a diagnosis of IPF; (2) final diagnostic agreement was higher between academic physicians compared with community physicians; (3) most diagnostic agreement occurred for cases of IPF; (4) most diagnostic discord occurred between cases of IPF versus HP, IPF versus NSIP, and the potential influence of CVD, with community-based physicians more likely to render a diagnosis of IPF. These data highlight how an individual patient with suspected DPLD can have a significantly different diagnosis depending on the physician and, particularly, the location of evaluation. Although a combined clinical, radiographic, and pathologic approach improves agreement, significant disagreement still exists. These data highlight the need for better ways to approach and classify patients with suspected DPLD.
Recent guidelines suggest that DPLDs, including IIPs, can be separated based on clinical, radiographic, and histopathologic criteria (16). The importance of “splitting” versus “lumping” DPLDs stems from the varied etiologies, treatments, and prognoses associated with different diseases. Academic physicians used a wider array of diagnoses compared with community-based physicians who used a more consistent diagnosis of IPF. In our series, 13 (33%) cases were believed to represent IPF by a majority of both community- and academic-based physicians. Importantly, community physicians made the diagnosis of IPF in 11 additional cases, where the academic physicians believed HP (n = 3), NSIP (n = 4), or CVD-associated (n = 4) disease was present. The importance of this finding is highlighted by the different prognoses among these diagnoses, and the novel therapeutic approaches currently under study based on biological plausibility in IPF. The relative minority of IPF cases compared with other IIPs in our series may reflect the requirement for all cases to have both an HRCT and SLB; cases of IPF based solely on definite HRCT criteria were excluded.
The current data document that the greatest disagreement in diagnosis occurred between academic and community physicians. However, significant disagreement was present even within academic centers. The better agreement for academic physicians likely reflects, at least in part, that these physicians with an interest in DPLD have collaborated on previous projects, including the generation of consensus statements. This suggests that more intense interaction between academic and community physicians could improve the diagnostic agreement between community and academic physicians, and should help standardize the approach to the treatment and study of patients with DPLD. In addition, the proportion of time devoted to clinical management of DPLD is likely important, as the community clinician who devoted the greatest time to the management of these disorders exhibited greater agreement with his academic counterparts.
We previously demonstrated that a dynamic, iterative approach to IIP diagnosis improves interobserver agreement among expert clinicians, particularly in patients without IPF (17). The current data highlight that a similar iterative approach improves diagnostic agreement within community clinicians. Two differences between community and academic physicians were evident. Community clinicians' agreement improved to a lesser degree than that of academic clinicians, and the final diagnosis by community pathologists was more influenced by the interaction with clinicians/radiologists than in the academic setting. This suggests that the final diagnosis in an academic setting is driven by pathology compared with the clinician/radiologist in the community setting. This latter observation could reflect the relative expertise, and thus assertiveness, of community clinicians/radiologists compared with community pathologists in the diagnosis of IIP. Pathology information appeared to influence the diagnosis of some cases in both community and academic centers, as the diagnosis of HP versus IPF seemed to correlate more with the presence/absence of granulomas compared with a clinical history of bird exposure.
Academic clinicians and radiologists used a wider array of diagnoses before receiving pathology information. The findings of subpleural, lower-lobe, honeycomb, and reticular change without micronodules, peribronchiolar nodules, consolidation, isolated cysts, or a predominance of ground glass opacity have a high positive predictive value for finding the histopathologic pattern of usual interstitial pneumonia (UIP) on SLB (10, 11, 22). A recent, survey-based study suggested that 67% of clinicians would accept an HRCT diagnosis of IPF, particularly if the observer had a higher self-rating of proficiency in reading HRCT (23). It is possible that the academic clinicians and radiologists in our study were more stringent in their application of these findings, and thus less likely to make a diagnosis of IPF without a biopsy.
Our prospectively collected data suggest that pathologists will consider clinical and radiologic data in rendering a final diagnosis and emphasizes the need for pathologists to consider these data in rendering a final diagnosis. This is supported by the decrease in intraobserver agreement between stages with and without clinical and radiologic information. This evaluative process was seen among both academic and community-based pathologists. Review of the individual patient data suggests that an exposure history consistent with HP, or a history suggestive of a CVC, was particularly likely to alter diagnosis, including away from a diagnosis of IPF. Given the difference in survival characteristic between IPF and connective tissue–associated UIP and chronic HP (21), this point may have important clinical ramifications.
A limitation of this study is the lack of transbronchial biopsy and/or bronchoalveolar lavage data for the majority of patients. This absence reflects the practice pattern at the University of Michigan and surrounding communities, where bronchoscopy is used infrequently when a diagnosis of IIP (especially UIP or NSIP) is considered. It is possible that rigorous collection of bronchoscopy data could impact the final diagnostic impression. Additional research is required to clarify the role of bronchoscopy, relative HRCT, and SLB in the diagnostic algorithm for patients with suspected IIP. Another limitation of this study is the involvement of academic physicians who devote the majority of their time to the study of interstitial lung disorders and, therefore, might not be representative of the whole “academic” physician group. This study was also mostly based in the United States and Europe, and might not fully represent the situation in other countries.
Our data expand on previous literature on interobserver agreement between clinicians, radiologists, and pathologists in diagnosing IIPs. We confirm that an interactive approach between clinicians, radiologists, and pathologists improves interobserver agreement. On the other hand, even with this approach, significant disagreement exists within, and particularly between, community and academic centers. The fact that community physicians were more likely to render a diagnosis of IPF has important implications, as individual patients with HP, NSIP, or CVD-associated ILD are more likely to respond to immunosuppressive treatment, whereas patients with IPF should be referred, whenever possible, for participation in therapeutic trials. Future efforts are needed to bridge the gap of apparent discordance between community and academic experts in their diagnostic proficiency. It is hoped that this will be accomplished with continued education, workshops, and increased interactions between academic and community-based physicians. In the short-term, these data suggest that, whenever possible, patients should be referred to centers with expertise in diffuse parenchymal lung disorders to help clarify the diagnosis and provide suggestions regarding treatment options.
The University of Michigan Fibrotic Lung Disease Network includes the following participants: University of Michigan, Division of Pulmonary and Critical Care, Ann Arbor, MI—D. Arenberg, W. Bria, D. Dahlgren, C. Grum, R. Hyzy, V. Lama, T. Ojo, M. Peters-Golden, R. Simon, T. Sisson, T. Standiford, V. Thannickal, E. White; Internal Medicine Clinic, Alpena, MI—P. Bachwich, C. Easton, J. Mazur; The Lung Center, Battle Creek, MI—S. Chaparala, G. Harrington, N. Potempa; Bay City, MI—S. Manawar, J. Summer; Clawson, MI—P. Hukku, J. Sung; Clinton Township, MI—R. Babcock; Pulmonary and Critical Care Medicine Consultants, Commerce, MI—J. Belen, M. Dunn, D. Maxwell, R. Reagle, R. Sherman, S. Simecek; Oakwood Hospital, Dearborn, MI—L. Victor; Henry Ford Hospital, Detroit, MI—B. DiGiovine, M. Eichenhorn, J. Popovich, Jr., D. Spizarny; Botsford General Hospital, Farmington Hills, MI—B. Rabinowitz; Pulmonary and Critical Care Specialists, Farmington Hills, MI—G. Ferguson, P. Kaplan, S. Sklar, W. VanderRoest; Pulmonary Associates, P.C., Flint, MI—O. Filos, V. Rao, M.V. Thomas, J. Varghese, J. Vyskocil, F. Wadenstorer; Grand Valley Internal Medicine, Grand Rapids, MI—J. Cantor, W. Katz, R. Johnson, Jr., D. Listello, J. Wilt; Michigan Medical Professional Company, Grand Rapids, MI—C. Acharya, W. Couwenhoven, T. Daum, M. Harrison, M. Koets, G. Sandman, G. VanOtteren; Michigan Medical, P.C., Holland, MI—S. Kraker; Huntington Woods, MI—M. Greenberger, A. O'Neill, D. Wu; Pulmonary Clinics of Southern Michigan, Jackson, MI—R.C. Albertson, III, J. Chauncey, T. Murray, G. Patten; Associated Pulmonary and Critical Care Specialists, P.C., Kalamazoo, MI—T. Abraham, J. Dirks, B. Dykstra, G. Grambau, J. Schoell; Pulmonary and Critical Care Associates, P.C., Kalamazoo, MI—R. Brush, S. Jefferson, J. Miller, S. Schuldheisz, M. Warlick; Pulmonary and Critical Care Consultants, Lansing, MI—J. Armstrong, A. Atkinson, T. Kantra, L. Rawsthorne, D. Young; Pulmonary Services, Lansing, MI—A. Abbasi, C.M. Gera, G. Kashyap, J. Morlock; Respiratory Medicine, Marquette, MI—S. Danek, A. Saari; Midland, MI—S. Yadam; Central Michigan Healthcare System, Mt. Pleasant, MI—E. Obeid; Muskegon Pulmonary Associates, Muskegon, MI—D. Hoch, A. Kleaveland; Owosso Medical Group, Owosso, MI—A. Allam, M.A. Gad, Jr.; Lung Associates, Pontiac, MI—A. Desai U. Dhanjal, A. Sethi; St. Joseph's Hospital, Pontiac, MI—F. Ahmad, L. Kaiser, L. Rosenthal, D. Sak; Physician HealthCare Network, Port Huron, MI—R. Ailani, M. Basha, A. Hadar, S. Holstine; Pulmonary, Critical Care, and Sleep, P.C., Rochester Hills, MI—M.W. Al-Ameri, R. Go, M. Kashlan; Rochester, MI—K. Aggarwal; Roseville, MI—W. Hanna, R. Marchese; William Beaumont Hospital, Royal Oak, MI—R. Begle, D. Erb, K.P. Ravikrishnan, J. Seidman, S. Sherman; Saginaw, MI—R. Agarwal, F. Ansari, T. Damuth, C. Indira; Spring Lake, MI—M. Ivey; Lakeside Healthcare Specialists, St. Joseph, MI—S. Deskins, A. Palmer, S. Shastri; Pulmonary and Critical Care Associates, St. Clair Shores, MI and Troy, MI—R. DiLisio, S. Galens, K. Grady, D. Harrington, R. Herbert, C. Hughes, J. Lee, A. Starrico, K. Stevens, M. Trunsky, W. Ventimiglia; Taylor, MI—D. Mahajan; Pulmonary Medicine Associates, Warren, MI—H. Kaplan, L. Tankanow; Henry Ford Wyandotte Hospital, Wyandotte, MI—M. Pensler; Toledo Pulmonary and Sleep Specialists, Toledo, OH—F.O. Horton, III, A. Nathanson, R. Wainz.
Supported in part by National Institutes of Health, National Heart, Lung, and Blood Institute grants P50HL-56402, NHLBI 2 K24 HL04212, and 1 K 23 HL68713.
This article has an online supplement, which is accessible from this issue's table of contents at www.atsjournals.org
Originally Published in Press as DOI: 10.1164/rccm.200606-833OC on January 25, 2007
Conflict of Interest Statement: K.R.F. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. A.-C.A. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. T.E.K. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. G.R. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. T.V.C. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. A.W. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. N.B. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. K.B. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. R.d.B. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. A.F. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. S.E.G. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. B.H.G. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. E.A.K. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. R.K. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. E.L. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. D.L. received less than $5,000 in 2004, 2005, and 2006 from Intermune, Inc., for interpretation of computed tomography scans, and has also received less than $5,000 from Encysive, Inc., for consultation regarding clinical trials; D.L. received $6,000 in 2006 for service on an advisory board for Actelion, Inc. A.G.N. received $2,500 for reviewing slides for a multicenter trial in 2005 for Intermune Ltd., and £9,500 for reviewing slides for a multicenter trial for Actelion Ltd. in 2006. J.Q. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. V.J.T. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. W.D.T. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. J.V. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. F.A.W. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. J.W. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. G.B.T. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. S.M. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. F.J.M. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript.