|Home | About | Journals | Submit | Contact Us | Français|
Assessment of the size and function of the functional single ventricle (FSV) is a key element in the management of patients following the Fontan procedure. Measurement variability of ventricular mass, volume and ejection fraction between observers by echocardiography and CMR and their reproducibility between readers in these patients has not been described. From the 546 patients enrolled in the Pediatric Heart Network Fontan Cross-Sectional Study (mean age 11.9±3.4 years), 100 echocardiograms and 50 CMR studies were assessed for measurement reproducibility; 124 subjects with paired studies were selected for comparison between modalities. Inter-observer agreement for qualitative grading of ventricular function by echocardiography was modest for left ventricular (LV) morphology (kappa= 0.42) and weak for right ventricular (RV) morphology (kappa= 0.12). For quantitative assessment, high intra-class correlation coefficients (ICC) were found for echocardiographic inter-observer (LV= 0.87–0.92; RV= 0.82–0.85) agreement of systolic and diastolic volumes, respectively. In contrast, ICCs for LV and RV mass were moderate (LV= 0.78; RV= 0.72). The corresponding ICCs by CMR were high (LV= 0.96; RV= 0.85). Volumes by echocardiography averaged 70% of CMR values. Interobserver reproducibility of EF was similar for both modalities. Although the absolute mean difference between modalities for ejection fraction was small (<2%), 95% limits of agreement were wide. In conclusion, agreement between observers of qualitative FSV function by echocardiography is modest. Measurements of FSV volume by 2D echocardiography underestimate CMR measurements but their reproducibility is high. Echocardiographic and CMR measurements of FSV EF demonstrate similar interobserver reproducibility whereas measurements of FSV mass and LV diastolic volume are more reproducible by CMR.
The NHLBI-sponsored Pediatric Heart Network Fontan Cross-Sectional Study was a prospective multicenter study designed to evaluate the relationship between health status and clinical measures in patients with functional single ventricle (FSV) who had undergone a Fontan procedure for palliation of congenital heart disease.1 As part of this study, echocardiographic and cardiac magnetic resonance imaging (CMR) evaluations of FSV size and function were performed. The purposes of this report were to define observer-related variability of echocardiographic and CMR-derived measures of ventricular mass, volume, and ejection fraction, and to determine the level of agreement for these measures between modalities.
A detailed description of the Pediatric Heart Network Fontan cross-sectional study design and its inclusion and exclusion criteria has been published.2 Briefly, subjects between the ages of 6 and 18 years were enrolled from March 2003 through April 2004 in 7 pediatric clinical centers in the United States and Canada. Prospective data collection for each subject occurred within a 3-month period and included health status questionnaires, two-dimensional (2D) and Doppler echocardiography, CMR, exercise, and other laboratory tests. The Institutional Review Board of each center approved the study protocol and written informed consent and assent were obtained according to local guidelines.
Two-dimensional and Doppler echocardiograms were obtained at participating centers according to the study protocol. None were performed under sedation. The studies were forwarded by the Data Coordinating Center to the Echocardiographic Core Laboratory for analysis by one of two echocardiographers (RM, MLS). Studies judged as acceptable for analysis were assigned an image quality grade: fair, good, or excellent based on subjective assessment of short- and long-axis images. Ventricular morphology was characterized as LV-dominant (e.g., tricuspid atresia), RV-dominant (e.g., hypoplastic left heart syndrome), or mixed (e.g., unbalanced atrioventricular canal defect).
The FSV was analyzed from the apical (ventricular long-axis) and parasternal short-axis views. The endocardial border of the FSV was traced at end-diastole and end-systole and the epicardial border was traced at end-diastole in both planes. End-diastolic volume (EDV), end-systolic volume (ESV), and mass were calculated using a biplane-modified Simpson rule.3 Ejection fraction was calculated as ([end-diastolic volume − end-systolic volume]/end-diastolic volume). Ventricular mass was calculated as myocardial end-diastolic volume (epicardial volume − endocardial volume) × myocardial density (1.05 g/ml). For the mixed morphology group, the volume and mass of each ventricle were measured separately, and the values for each ventricle were included in their respective morphologic groups. Global ventricular systolic function was qualitatively graded as normal function, mild, moderate, or severe dysfunction. Echocardiographic data were reviewed and measurements made using custom software (Marcus Laboratories, Boston, MA).
CMR studies were performed in each participating center on a 1.5 T scanner using a standard imaging protocol. Study subjects did not undergo CMR as part of their assessment if they met any of the following criteria: 1) unable to cooperate without sedation; 2) had a pacemaker, defibrillator, permanent pacemaker lead, or implanted device considered a contraindication according to institutional guidelines; 3) had intravascular occlusion coils deemed to result in excessive image artifact; or 4) were <6 weeks from endovascular device implantation.
The standardized imaging protocol included ECG-gated segmented k-space fast (turbo) gradient (14% of studies) or steady state free precession (86% of studies) cine MR acquisitions in the vertical and horizontal long-axis planes, and contiguous short-axis cine imaging from the atrioventricular junction through the cardiac apex. Deidentified CMR data were analyzed using commercially available software (MASS, Medis, Leiden, The Netherlands)at the Core CMR laboratory. Left and/or right ventricular end-diastolic (maximal) and end-systolic (minimal) volumes, and mass at end-diastole were measured in the ventricular short-axis plane as described by Lorenz.4 Stroke volumes, ejection fraction (EF), and mass-to-volume ratio were calculated.
To evaluate inter- and intra-observer variability for echocardiograms and inter-observer variability for CMR, eligible subjects were randomly selected for repeat analysis. To be eligible, the initial study was required to be rated as acceptable (at least fair quality) and have complete assessment of FSV volumes and ejection fractions by the initial reader. Of the 546 subjects enrolled in the study, 404 (74%) patients had echocardiograms and 159 (29%) had CMR studies that met these criteria. From this pool, 100 echocardiograms and 50 CMR studies were randomly selected for observer variability analysis. Sampling was stratified based on ventricular morphology type and echocardiographic image quality grade. The echocardiographic analysis dataset consisted of 3 readings per echocardiogram – two by the same core lab reviewer for intra-observer variability and one by the other reader for inter-observer variability. The CMR dataset consisted of two readings per CMR study, the original core lab review (TG), and repeat evaluation (AP) for interobserver variability. Intra-observer variability was not assessed for CMR because the relatively small number of studies resulted in a degree a familiarity with the images such that repeat contouring would not have resulted in a de-novo analysis.
To compare between echocardiographic and CMR measurements of mass, volumes, and ejection fraction, all of the 124 subjects with complete datasets of both modalities rated as acceptable were selected.
Kappa statistics with Cicchetti-Allison weights and frequencies of concordance were calculated for assessment of intra- and inter-observer agreement for echocardiographic image quality and FSV function grade. This analysis was performed only when both readers agreed that the structure was present and evaluable. Weighted kappa statistics were used to determine the agreement between echocardiographic quantitative and qualitative approaches to FSV systolic function grade using CMR-based FSV function grade as reference standard.
The intra- and inter-observer variability of continuous variables was assessed using the intra-class correlation coefficient (ICC) estimated with variance component models.5 The ICC can be interpreted as the proportion of variability explained by subject differences as opposed to rater differences and random error. To meet the normality and constant variance assumptions of this model, log transformations were used for EDV, ESV, and mass. Because estimation of ICC can be imprecise when the number of raters is small, within-subject standard deviation (SD) was also used to assess intra- and inter-observer variability.6 Plots of differences versus mean values were produced to graphically examine variability and magnitude of the differences. Likelihood ratio tests (LRT) were used to test differences in inter-observer within-subject SD between echocardiography and CMR and between subgroups stratified by image quality grade. Bland-Altman 95% limits of agreement with log transformation were calculated to assess agreement between echocardiographic and CMR measurements. To examine the relationship between level of agreement and echocardiographic image quality, geometric mean ratios and 95% limits of agreement were calculated to assess agreement between echocardiographic and CMR measurements. Log transformations were used and geometric mean ratios reported for EDV, ESV, and mass. One outlier was excluded from CMR RV ESV and RV EF inter-observer analyses. For all analyses, differences were considered to be significant for p-value <0.05. Statistical analysis was performed using SAS 9.1.3 (Cary, NC) and figures were produced using S-Plus 6.2 (Insightful Corp., Seattle, WA).
The demographic, anatomic, and image quality data for the echocardiography and CMR groups are summarized in Table 1. The groups were similar with respect to age, gender, race, and ventricular type.
Table 2 summarizes intra- and inter-observer agreement for echocardiographic image quality and ventricular function grade. Although the confidence intervals are wide, the data suggest agreement for overall image quality grade was weak, with readers’ assessments of quality being concordant less than half the time. Agreement regarding FSV function grade was moderate for LV morphology and weak for RV morphology.
FSV systolic function was categorized by echocardiography as normal, mild, moderate, or severe dysfunction based on EF measurements (quantitative approach) and by qualitative assessment (“eyeball” approach). For the quantitative approach, EF was classified as normal (>55%), mild (41–55%), moderate (31–40%), or severe (≤30%) dysfunction and for the qualitative approach the same categories were employed. CMR-based FSV function grade was used as reference standard, also using the same quantitative categorization. The agreement in FSV function grade between categorization based on quantitative echocardiographic measurements and CMR was weak (weighted kappa [95% CI]: LV = 0.13 [−0.03 to 0.30], 58% concordant; RV = 0.34 [0.07 to 0.61], 60% concordant). The agreement in FSV function grade between categorization based on qualitative echocardiographic assessment and CMR was similarly weak (weighted kappa [95% CI]: LV = 0.28 [0.09 to 0.46], 65% concordant; RV = 0.30 [0.144 to 0.45], 61% concordant). When only studies with echocardiographic image quality rated as good or excellent were included in the analysis, the level of agreement was slightly improved for the qualitative approach (weighted kappa [95% CI]: LV = 0.33 [0.14 to 0.51], 66% concordant, RV = 0.32 [0.13 to 0.51], 63% concordant) but did not improve for the quantitative approach (weighted kappa [95% CI]: LV = 0.17 [−0.01 to 0.34], 57% concordant; [RV = 0.25 [−0.03 to 0.52], 61% concordant).
Table 3 summarizes intra- and inter-observer variability for echocardiographic measurements and inter-observer variability data for CMR. Plots of differences versus mean values demonstrating variability and magnitude of the differences between readers are shown in Figures 1–3. Intra-observer agreement was high for echocardiographic measurements of LV volumes and for RV volumes. Intra-observer agreement was moderate for FSV mass and modest for ejection fraction. Patterns for inter-observer echocardiographic variability mirrored those for intra-observer variability with lower or similar ICCs.
For CMR measurements of LV and RV volumes the level of agreement between readers was high. High level of inter-observer agreement was also noted for LV and RV mass measurement but, similar to echocardiography, the agreement of EF was modest. Compared to echocardiography, CMR within-subject standard deviations tended to be lower, reaching statistical significance for LV EDV, LV mass, and RV mass (Table 3).
The comparison between echocardiographic- and CMR-derived measurements is summarized in Table 4 and Figures 4 and and5.5. Compared with CMR, echocardiographic measurements of ventricular volume were smaller (70–79%) and the limits of agreement were wide. For example, echocardiographic measurement of LV ESV would range from approximately a quarter to twice that of the CMR measurement. Although on average the measurements of ejection fraction and mass were similar between modalities (e.g., mean difference of 1.6% for LV EF and 0.2% for RV EF), the limits of agreement were wide (Table 4, Figure 5).
The results of this study reveal several important findings regarding the non-invasive assessment of FSV size and function. While qualitative assessment of LV function by 2D echocardiography is moderately reproducible between observers, assessment of the RV is poorly reproducible. In contrast, quantitative echocardiographic measures of ventricular volumes are quite reproducible for both LV and RV morphologies. Even though 2D echocardiographic measurements systematically underestimated CMR-derived ventricular volume measurements, the reproducibility of EF measurements is comparable by both modalities. Interobserver reproducibility of CMR measurements is generally better than echocardiography, reaching statistical significance for measurements of LV EDV and for LV and RV mass. Finally, both qualitative and quantitative echocardiographic assessments of FSV function grade agree relatively weakly with CMR with no clear advantage to either approach.
The limitations inherent to 2D as opposed to 3-dimensional (3D) imaging techniques have been demonstrated by Chuang at al.7 These authors showed that biplane 2D measurements of LV volume and ejection fraction were less accurate and reproducible than 3D volumetric measurements, regardless of imaging modality—echocardiography or CMR in adult patients with dilated cardiomyopathy. Difficulties in acquiring “true” long- and short-axis imaging planes by 2D techniques were felt to be a key weakness of the biplane method. This limitation has been shown to be even more pronounced in the assessment of RV volume where prediction of volume based on linear and cross-sectional measurements is more difficult.8–10 Extrapolation from these studies to the functional single ventricle should yield similar results, as chamber geometry and its orientation within the thorax are often unpredictable and imaging of the anterior, retrosternal free wall is often challenging. Although 3D echocardiography holds promise for a more accurate assessment of ventricular volume in a biventricular circulation,11–15 its accuracy and reproducibility in patients with FSV awaits confirmation. Soriano et al. recently demonstrated that measurements of FSV volumes, EF, and mass by 3D echocardiography correlate well with CMR measurements.16 However their study included a relatively small number of young patients (29 infants; median age 7 months) whose studies were performed under general anesthesia. Reproducibility of 3D echocardiographic measurements of FSV size and function in older patients with all forms of FSV requires further validation. Another alternative to 2D echocardiography for the assessment of ventricular function is based on Doppler techniques.17,18 However, as with 3D echocardiography, the precise role of these techniques in patients that have undergone Fontan palliation awaits further investigation.
Although this report demonstrates low agreement for qualitative assessment of RV systolic function, this method is in prevalent use in most echocardiographic laboratories. Poor reproducibility of qualitative assessment of RV function was also demonstrated in patients with right-sided congenital heart disease and two-ventricle circulation,19 but has not been reported in single ventricle physiology.
Lower reproducibility of FSV ejection fraction measurements was noted for both echocardiography and CMR despite high ICC of end-diastolic and end-systolic volume measurements. The most likely explanation is that the variability of measuring individual parameters (EDV and ESV) is magnified when they are subtracted and then divided. Similar observations were made in the studies of Lipshultz et al. (echocardiography) and Mooij et al. (CMR).20,21
The results of this study are in agreement with previous reports that demonstrated the accuracy and reproducibility of CMR measurements of ventricular volume, ejection fraction, and mass.21–24 However, several practical limitations restrict the use of CMR in patients after the Fontan procedure. Presence of a pacemaker or cardiac defibrillator (13% in this cohort) is considered a strong relative contraindication for CMR imaging. Image artifacts from metallic implants precluded quantitative volumetric analysis in 20% of patients with Fontan palliation reported by Garg et al. 25 and were the primary reason for a lower image quality score in this cohort. Overall, the CMR data were inadequate or incomplete in 30% of patients in whom the test was performed, predominantly due to metallic artifacts. However, in patients who are able to cooperate with the examination, have no metallic artifacts that obscure the ventricular mass, and have no contraindications to CMR, this modality offers an advantage in terms of reproducible assessment of ventricular size and function. Moreover, the lower inter-observer standard deviations of some CMR-derived measurements (LVEDV, LV and RV mass) found in this study suggest that the sample size required to demonstrate a treatment effect or a change over time would be smaller with CMR as compared with 2D echocardiography.24
Several limitations of this study merit attention. Our findings are conditional on having both an acceptable CMR and echocardiographic studies for analysis. In this highly heterogeneous population, the success rate for obtaining adequate quality studies may vary by modality due to a variety of patient- and operator-related factors. CMR reproducibility may have been affected by the use of two cine MRI techniques, although the percentage of non-SSFP utilization was small. The number of patients with systolic ventricular dysfunction was small, which limited our ability to detect a possible trend between echocardiography-CMR agreement and ejection fraction. The sample size for subgroup analyses was small, which might have precluded detection of subtle associations with image quality. It should be noted that based on inclusion criteria, the study group included only patients between the ages of 6 and 18 years. Extrapolation to older patients may not be accurate due to worsening acoustic windows and a higher prevalence of ventricular dysfunction. Extrapolation to younger patients may also not be accurate due to potentially better acoustic windows. Nonetheless, the results of this study are derived from a contemporary cohort of young patients studied at 7 different institutions, representing a relatively large group of patients with Fontan palliation.
Supported by U01 grants from the National Heart, Lung, and Blood Institute (HL068269, HL068270, HL068279, HL068281, HL068285, HL068292, HL068290, HL068288).
Gail Pearson, Mario Stylianou, Judith Massicot-Fisher, Marsha Mathis, Victoria Pemberton
New England Research Institutes, Lynn Sleeper (PI), Steven Colan, Dianne Gallagher, Patti Nash, Lisa Wruck, Minmin Lu
Lynn Mahony, University of Texas Southwestern Medical Center
Children’s Hospital Boston, Jane Newburger (PI), Stephen Roth, Roger Breitbart, Jonathan Rhodes, Jodi Elder, Ellen McGrath; Children’s Hospital of New York, Welton M. Gersony (PI), Seema Mital, Beth Printz, Ashwin Prakash, Darlene Servedio; Children’s Hospital of Philadelphia, Victoria Vetter (PI), Bernard J. Clark, Mark Fogel, Steven Paridon, Jack Rychik, Margaret Harkins, Jamie Koh; Duke University, Page A. W. Anderson (PI), Rene Herlong, Lynne Hurwitz, Jennifer S. Li, Ann Marie Nawrocki; Medical University of South Carolina, J. Philip Saul (PI), Andrew M. Atz, Andrew D. Blaufox, Girish Shirali, Jon Lucas, Amy Blevins; Primary Children’s Medical Center, Salt Lake City, Utah, LuAnn Minich (PI), Richard Williams, Linda Lambert, Michael Puchalski; Hospital for Sick Children, Toronto, Brian McCrindle (PI), Timothy Bradley, Kevin Roman, Jennifer Russell, Shi-Joon Yoo, Elizabeth Radojewski, Nancy Slater
Cardiac MRI, Children’s Hospital Boston: Tal Geva (Director); Andrew J. Powell Echocardiography, Children’s Hospital Boston: Steven Colan (Director), Marcy Schwartz, Renee Margossian
Michael Artman, Chair; Dana Connolly, Timothy Feltes, Julie Johnson, Jeffrey Krischer, G. Paul Matherne.
John Kugler, Chair; Kathryn Davis, David J. Driscoll, Mark Galantowicz, Sally A. Hunsberger, Thomas J. Knight, Catherine L. Webb, Lawrence Wissow.
The authors state no conflicts of interest exist.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.