|Home | About | Journals | Submit | Contact Us | Français|
To describe the methods used for assessment of baseline fundus characteristics from color photography and fluorescein angiography (FA) in the Comparison of the Age-Related Macular Degeneration Treatments Trials (CATT), and the relationship between these characteristics and visual acuity.
Randomized, masked, multicenter trial.
This investigation included 1185 participants of the CATT study.
Baseline stereoscopic color fundus photographs and FAs of participants in the CATT study were assessed at a central fundus photograph reading center by masked readers. Replicate assessments of random samples of photographs were performed to assess intra- and inter-grader agreements. The association of the lesion characteristics with baseline visual acuity was assessed using analyses of variance and correlation coefficients.
Intra- and inter-grader reproducibility, visual acuity and lesion characteristics.
Intra- and inter-grader reproducibility showed agreements ranging from 75% to 100% and weighted kappas ranging from 0.48 to 1.0 for qualitative determinations. The intra-class correlation coefficients were 0.96-0.97 for quantitative measurements of choroidal neovascularization (CNV) area and total area of CNV lesion. The mean visual acuity (SE) varied by the type of pathology in the foveal center: 64.5 (0.7) letters for fluid only, 59.0 (0.5) for CNV, and 58.7 (1.3) for hemorrhage (p<0.001). Fibrotic or atrophic scar present in the lesion, but not under the center of the fovea, was also associated with a markedly reduced visual acuity 48.4 (2.2), p<0.0001. Although total area of CNV lesion was weakly correlated with visual acuity when all participants were assessed (Spearman correlation coefficient ρ= −0.16, p<0.001), the correlation was stronger within patients with predominantly classic lesions (ρ=−0.42, p<0.001).
Our results show that the methodology used for grading CATT fundus images has good reproducibility. As expected, larger total CNV lesion area, and pathologic findings such as hemorrhage, fibrosis and atrophy at baseline are associated with decreased visual acuity.
The authors have no proprietary or commercial interests in any materials discussed in this article.
The Comparison of age related macular degeneration (AMD) Treatments Trials (CATT) is a randomized clinical trial designed to compare the efficacy and safety of ranibizumab and bevacizumab and to investigate whether less than monthly dosing compromises long-term visual acuity.1
All color photographs and fluorescein angiograms collected during the study were assessed at the fundus photography reading center located at the Department of Ophthalmology of the Perlman School of Medicine of the University of Pennsylvania. The purposes of this manuscript are: a) to review the methodology used for grading these photographs; b) to describe the baseline fundus morphologic and fluorescein angiographic characteristics of the CATT participants; and 3) to evaluate the association between morphological features and visual acuity.
Between February 2008 and December 2009, a total of 1185 patients were enrolled through 43 clinical centers in the United States. Inclusion criteria included: age ≥ 50 years, presence of previously untreated active choroidal neovascularization (CNV) secondary to age-related macular degeneration (AMD) in the study eye, and visual acuity between 20/25 and 20/320 (letter score of 23 – 82 on electronic visual acuity testing).2 Both leakage on fluorescein angiography and fluid on optical coherence tomography (OCT), located either within or below the retina or below the retinal pigment epithelium, were required to establish the presence of active CNV. CNV or its sequelae such as fluid, pigment epithelial detachment or hemorrhage needed to be under the fovea. The total area of retinal fibrosis could not exceed 50% of the total lesion. Although patients with hemorrhage involving more than 50% of the total lesion area were initially excluded from the trial, this exclusion criterion was later eliminated allowing patients with hemorrhage larger than 50% to enroll in the study. One or more drusen (>63 microns) had to be present in either eye or late AMD had to be present in the fellow eye. The study was approved by an institutional review board associated with each center. All patients provided written informed consent. The study was compliant with Health Insurance Portability and Accountability Act regulations. The CATT trial was registered with ClinicalTrials.gov (NCT00593450).
Patients with previous treatment for neovascular AMD in the study eye, patients who are actively receiving intravenous bevacizumab, or patients receiving treatment with any investigational drug or device likely to have ocular effects were ineligible. Ocular exclusion criteria included: fibrosis or geographic atrophy involving the center of the fovea; CNV in either eye due to causes other than AMD such as ocular histoplasmosis, trauma or pathologic myopia; retinal pigment epithelial tear; any concurrent intraocular conditions that could require medical or surgical intervention during the 2 years of the study; patients with other progressive retinal disease likely to affect visual acuity within 2 years. Patients with pattern dystrophy with CNV and drusen determined to have definite AMD were deemed eligible3.
Mydriatic stereoscopic color photographs and fluorescein angiograms of the macula were obtained from both eyes of each participant using standardized protocols. Stereo photographic images were acquired digitally in almost all cases. In 21 cases from two sites, photographic film was used for color photographs. Stereo pair color and red free images of the disc (modified field 1) and macula (field 2) were obtained for each eye. Fluorescein angiography stereo pairs of the study eye were obtained during the early (15 sec to 45 sec, 5-8 pairs), mid (1 to 3 minutes, 3 pairs) and late (1 pair at each of 5 and 10 minutes) phase and of the non-study eye at 2 and 10 minutes. All digital imaging systems at the Clinical Centers were certified by the CATT Fundus Photography Reading Center (CATT FPRC) before patient enrollment was started. Topcon IMAGENet System, Ophthalmic Imaging System (OIS) WinStation, MRP Ophtha Vision system, Escalon or Zeiss Visupac systems were acceptable. An alternate ophthalmic digital system was approved if all requirements for acquisition, archiving, magnification, image quality, and image accessibility by the reading center were met. The entire angiogram and color photographs were written to a CD using CATT custom developed submission application software. The submission application checked for image resolution, bit depth and other certified image parameters and had filtering capabilities that prompted for missing and incomplete entries.
The fundus digital images were viewed on dual LCD color monitors using software applications which included methods for rendering stereoscopic images, image comparison, and measurements. In order to achieve comparable grading results at multiple workstations, the viewing monitors were calibrated and standardized for brightness and color characteristic. Original images were not enhanced or otherwise altered. Film based images were viewed in stereo on a light box with 5× Donaldson stereo viewers.
All graders were masked to the treatment assignment of the patient. A multi-step grading procedure was used for photographic assessment. The first step assessed eligibility and photographic quality. The eligibility assessment determined whether: a) the fluorescein angiogram images were of sufficient quality to determine eligibility; b) there was active leakage of fluorescein on the angiogram; c) either CNV or sequelae of CNV, such as pigment epithelial detachment, hemorrhage, blocked fluorescence, macular edema or fluid involving the center of the fovea were present; d) the area of fibrosis was less than 50% of the total lesion; e) no fibrosis or geographic atrophy was present in the foveal center; f) no retinal pigment tear was present; g) the CNV was not due to causes other than AMD; and h) there was no progressive retinal disease that might affect vision.
After the patient was enrolled by the clinical sites, an eligibility grading was done independently by a grader and the Director of the Reading Center and recorded on baseline eligibility evaluation forms. Discrepancies on the two forms were adjudicated between the grader and the Director of the Reading Center and a final consensus eligibility form was completed. In cases where persisting discrepancies existed, the images were reviewed by the Principal Investigator of the Reading Center, and then, the consensus form was completed. Only the consensus forms were data entered into the CATT Reading Center data base. Ineligible cases were reported back to the clinical sites. After initial grading of eligibility and photographic quality, a detailed grading of the CNV and total CNV lesion were performed following a similar adjudication and review process.
The type of CNV, retinal hemorrhages, fluid, serous pigment epithelial detachment (SPED), and atrophic or fibrotic scars were identified using previously outlined descriptions4. There were no limitations on the shape and size of geographic atrophy. Features commonly associated with retinal angiomatous proliferans (RAP) lesions such as hot spot (focal area of hyperfluorescense), superficial hemorrhage, lipid, serous pigment epithelial detachment (SPED), fibrovascular PED and retinal and choroidal vessel anastomosis were documented. The presence of a hotspot on fluorescein angiography was required for a lesion to be identified as RAP lesion5,6,7. Blocked fluorescence was diagnosed only if it was not related to increased pigmentation or hemorrhage. A finding was considered present or absent if the decision reflected 80% or greater certainty. Otherwise, lesion presence or absence was graded as questionable. A decision of “cannot grade” was made if other fundus pathology, photographic quality, or artifact obscured the object of interest in a way that a definitive decision could not be made with 80% certainty.
Graders were trained for a period of 3 months to perform systematic evaluations of the color photographs and fluorescein angiograms of eyes with AMD. Training was also aimed at achieving consistent grader agreement in the identification of key features such as: leakage from the CNV, hemorrhage, fibrotic scar, fluid, SPED, RPE tear, geographic atrophy and RAP. Grade/re-grade reproducibility was assessed in 84 participants for qualitative gradings and in 24 participants for quantitative determinations.
Measurements of CNV area were done using Image J, a public domain Java image processing program developed by NIH, which is available as free software from http://rsbweb.nih.gov/ij/ Accessed July 7, 2009. CNV and total CNV lesion (TCNVL) area were measured. The TCNVL area included the CNV plus contiguous hemorrhage, SPED, atophic scar, fibrotic scar, and blocked fluorescence.
Because the fundus cameras used in the study had different magnifications, we used the University of Wisconsin Fundus Photograph Reading Center Pragmatic Calibration method which accounts for differences in photographic settings (Invest Ophthalmol Vis Sci 49 [Suppl]: 2243, 2008. For each participant’s visit, an image clearly showing the optic disc center and the center of the fovea was chosen and a line drawn between these two points. The distance was assumed to be 4.5 mm and a calibration factor was generated and applied to all the images of that eye.
Images were independently measured by two graders using dual monitors. One monitor displayed the fluorescein angiogram image used for measurement and the other displayed the color images, red-free images and all the flourescein angiogram images obtained during that visit.
One high quality fluorescein angiogram image was chosen for drawing the extent of the lesion. Typically, for classic CNV, the image just prior to the onset of leakage was chosen. For occult CNV, a later image showing maximum staining was chosen. For mixed CNV lesions, where the full extent of the occult component was visible only in later frames of the angiogram, a later image was chosen.
Using the freehand selections tool provided by the software, the outline of the CNV was drawn. The output of the total measurement in millimeters-square was recorded and the image saved. The procedure was repeated for measurement of the TCNVL area on the same image. The area in mm2 was converted to disc area (DA) by dividing by 2.54. The contours of the drawing outlines were as circumferential as possible inclusive of outcroppings of CNV larger than the diameter of the largest vein at the disc margin. In rare instances showing more than one discrete CNV lesion, the lesion closer to the foveal center was measured.
While measuring a RAP lesion without associated larger areas of occult or classic CNV, the area of the hot spot in its widest appearance and any contiguous intra-retinal hemorrhage, were measured.
In some instances measurements could not be done because of poor quality photographs, presence of leakage from undetermined source at the edge of GA, and in relatively flat occult lesions whose borders merged with areas of non specific atrophy. In cases where lesions extended beyond the image edge, the measurement included the area up to the edge of the image.
Adjudication was done if the percent difference between two graders was > 50% or the absolute value of the difference in the area was > 3.0 mm2. In addition, adjudications were also performed if one grader could not grade an image and the other grader made a measurement. The graders had access to their original drawings and data forms during adjudication.
The refraction and visual acuity testing protocol designed by the Diabetic Retinopathy Clinical Research Network (DRCRnet, 2005) was used during the study.8
All analyses were conducted using SAS Version 9.2 (SAS Institute Inc., Cary NC). Descriptive analyses were performed to summarize the baseline lesion features, CNV area, TCNVL area, and visual acuity score. Grade/re-grade agreements were evaluated on four random samples of images chosen for re-grade of lesion features (n=84 eyes total), and one random sample of images chosen for re-measure of the CNV size and lesion type (n=24). Percentage agreement and weighted kappa statistics with confidence intervals were calculated for grade/re-grade agreement based on the consensus grading. Pair-wise percent agreement and multi-grader (2 to 3 graders) kappas with bootstrapped confidence intervals were calculated for assessing the inter-grader agreement9. For grade/re-grade and inter-grader agreement for CNV area measurements, intraclass correlations with mean difference and 95% limit of agreement were calculated.
Analyses of variance (ANOVA) were used to assess the association of baseline lesion characteristics with visual acuity score. Study eyes without CNV or un-gradable on a particular characteristic are reported but not included in the statistical comparisons. CNV area measurements were categorized by quartile, with gradable but un-measureable CNV as a separate category. In addition to ANOVA comparisons of VA score among groups categorized by quartiles, correlations between visual acuity and lesion size were also assessed by Spearman correlation coefficient ρ due to the skewed distributions of lesion size measurements.
The correlation analyses were further stratified by lesion type. After the univariate analysis of each lesion characteristic with visual acuity, a multivariate linear regression model was performed by initially including all variables with p-values <0.10 from the univariate analysis. The multivariate model went through backward selection and only variables with p-value <0.05 were kept in the final multivariate model. An interaction between CNV size by quartile and lesion type was included in the multivariate model because the correlation coefficient between lesion area and visual acuity score varied across type of lesion.
Among 1185 patients enrolled in the study, 2.36% (28 patients) did not meet the photography eligibility criteria. The most common reason for ineligibility was absence of leakage on fluorescein angiography (Table 1). Other reasons included >50% of the lesion composed of fibrotic scars or hemorrhage, fibrosis or geographic atrophy in the foveal center, no sequelae of CNV under the fovea, and others (Table 1). Out of all baseline visit images, 96.7% were judged to be gradable.
Table 2 summarizes the intra- and inter-grader agreement for baseline features in the study eyes. Weighted Kappas for the consensus grade/re-grade agreement based on repeated measurements in 84 study eyes ranged from 0.48 to 1.0 and the percent agreement ranged between 75% and 100%. Inter-grader agreement showed Weighted Kappas ranging from 0.53 to 1.00 and % agreements ranging from 80% to 100%.
Intraclass correlations for lesion size measurements repeated in 23 study eyes were 0.96 for the consensus grade-re-grade agreement and 0.97 for the inter-grader agreement (Table 2). There was an error during re-grading of one additional image in which one grader measured 9.0 DA while the other measured 0.7 DA, for both CNV area and TCNVL area, and during consensus grading it was decided that 0.7 was correct. When all 24 participants were assessed together the agreement was 0.63 for the consensus grade-re-grade agreement and 0.65 for the inter-grader agreement.
Out of the 1185 study eyes, more than half had CNV under the center of the fovea, and a quarter of them had fluid only in the fovea (Table 3). Approximately one third of the eyes had contiguous hemorrhage as part of the lesion. Other components noted were blocked fluorescence in 172 (14.5%) eyes, serous pigment epithelial detachment (SPED) in 63 (5.3%) eyes, and fibrotic or atrophic scar that was not subfoveal 46 (3.9%). RAP lesions were present in 128 (10.8%) eyes and geographic atrophy that was not subfoveal was observed in 82 (6.9%) study eyes. More than half of the CNV were occult only lesions followed by predominantly classic CNV (22.5%) and minimally classic CNV (16.6%).
There was a strong association between the presence of several lesion components and mean visual acuity. Patients with hemorrhage had significantly worse visual acuity than those without hemorrhage (58.2 vs 61.7, p<0.001, Table 3). Larger areas of hemorrhage were associated with worse visual acuity (p<0.001). Eyes with blocked fluorescence had a significantly worse visual acuity than those without it (57.7 vs 60.9, p=0.004). Lesions that included fibrotic or atrophic scar components were also associated with a markedly reduced visual acuity (48.4 vs 60.9, p<0.0001). Although SPED was associated with mildly better visual acuity, the difference was not statistically significant.
Presence of pathology and the type of pathology in the foveal center were associated, as expected, with a significantly worse visual acuity (p<0.001). For example, eyes with only fluid in the fovea had mean acuity of 64.5 (0.7, SE) letters, eyes with CNV under the fovea had average acuity of 59 (0.5) letters, and those with hemorrhage had average acuity of 58.7 (1.3) letters. Presence of other lesion components under the fovea was associated with average acuity of 53.7 (2.3) letters, whereas eyes with SPED had a very similar acuity to those with fluid only.
The type of lesion was associated with statistically significant different average visual acuity (p<0.001, Table 3). Predominantly classic lesions, observed in 267 eyes (22.5%) had a mean visual acuity of 55.8 letters, minimally classic lesions, observed in 197 (16.6%), had a mean visual acuity of 57.2 letters, and occult only lesions, observed in 696 eyes (63.1%) had a mean visual acuity of 63.1 letters. RAP lesions present in 128 eyes (10.8%) were not associated with a significant difference in vision from other types of AMD lesions. Finally, presence of geographic atrophy (not under the foveal center) was associated with worse visual acuity although the difference was of borderline significance (p=0.07).
Larger area of CNV and TCNVL were both associated with lower average visual acuity (p=0.03 and p<0.0001, respectively, Table 3), with Spearman correlation coefficient ρ = −0.08 and −0.16 respectively (Table 4). Larger area of CNV was significantly associated with worse average visual acuity for predominantly classic (ρ = −0.42, p<0.001) and occult only (ρ= −0.11, p<0.01) lesions, but not for minimally classic lesions (ρ=−0.10, p=0.20, Table 3). Larger area of TCNVL was associated with worse acuity for predominantly classic (p<0.001), minimally classic (p=0.001) and occult lesions (p<0.001, Table 4).
The multivariate analysis showed that worse VA is associated with fibrotic or atrophic scar, presence of CNV in the fovea center, larger hemorrhage size, and presence of geographic atrophy (Table 5). A statistically significant interaction was detected between lesion type and area of CNV (p<0.0001). The association between larger area of CNV and worse VA was statistically significant for predominantly classic CNV (p<0.001), and was not statistically significant for minimally classic CNV (p=0.10) and occult CNV (p=0.15).
Study eyes in which CNV area and TCNVL area could not be determined tended to have worse visual acuity. About 90% of eyes in which CNV area could not be determined had photographs of good or fair quality, suggesting that the decreased vision in these eyes was most probably due to the complexity of the lesion and not only to decreased media clarity.
Our results show that lesion components, location of the CNV lesion, lesion characteristics, presence and extent of hemorrhage and area of CNV and TCNVL have a strong effect on baseline visual acuity. Not surprisingly, patients with CNV, hemorrhage or blocked fluorescence under the foveal center had significantly worse visual acuity than those that had only fluid under the fovea. Larger area of CNV and TCNVL were also associated with decreased visual acuity.
Eyes with predominantly classic CNV had worse baseline visual acuity than those with occult CNV and minimally classic CNV, a result that is consistent with the findings of the Treatment of AMD with Photodynamic Therapy (TAP) investigation in which untreated patients with predominantly classic lesions had a lower mean visual acuity letter score (51, approximate Snellen equivalent: 20/100) than untreated patients with minimally classic lesions (54, approximate Snellen equivalent: 20/80)10.
A possible explanation for this difference may be that classic CNV develops closer to the outer receptor layer than occult CNV, and therefore, may have a more deleterious effect on vision. In addition, both post-mortem and surgically excised tissue have shown classic CNV to have much larger caliber vessels.11,12 a factor that could lead to a larger disruption in the metabolism of the photoreceptors.
Although the replacement of laser, surgical and photodynamic therapies with the newer intravitreal anti-VEGF therapy has placed a lesser emphasis on the type of CNV defined by the Macular Photocoagulation Study 13, it is important to identify CNV characteristics at baseline to evaluate potential differences in risk factors and treatment efficacy. The MPS study reported, for example that classic CNV presented as smaller lesions than other types of CNV but had the worst visual acuity at enrollment14,15, a result that is in agreement with our finding showing that predominantly classic lesions have the worst visual acuity at baseline.
Different components of the CNV lesion had varied effects on visual acuity in our study. Eligibility criteria for enrollment into the CATT study required that fibrotic scars or atrophic scars had to be less than 50% of the total CNV lesion and could not be located under the foveal center. Less than 5% of the study eyes had scars associated with the lesion and these eyes showed the largest reduction in average visual acuity when compared to study eyes with other lesions components, corroborating results from other smaller studies.16,17
Eyes with geographic atrophy (which by eligibility criteria could not be under the fovea) had mildly decreased visual acuity that was not statistically significantly different from those of eyes without this feature. Finally, eyes with RAP lesions or SPED did not have a significantly different visual acuity from those of eyes without these characteristics.
The morphological baseline characteristics in the CATT cohort differ from those of participants of earlier anti-VEGF studies targeting AMD such as MARINA18 and ANCHOR19. While ANCHOR and MARINA required that the CNV should be present under the center of the fovea, one-fourth of our study eyes had only fluid in the center of the fovea. The MARINA18 cohort study included minimally classic or occult CNV lesions and the ANCHOR19 cohort included predominantly classic CNV lesions. The CATT cohort, on the other hand had patients of both categories with three-fourths of patients having occult or minimally classic lesions and the other quarter made up of predominantly classic CNV. In addition 7% of patients had hemorrhage larger than 50% of the CNV lesion and 11% of patient had RAP lesions both features that were not included in previous trials. Therefore, the CATT cohort baseline retinal images consisted of a diversity of morphological features that more closely represents the findings seen in a population of newly diagnosed wet AMD patients.
Grading of the fundus morphological features on the color photographs and fluorescein angiograms in this study was done at the University of Pennsylvania FPRC while concurrent grading of the OCT scans was performed at the Duke Reading Center. Study entry eligibility criteria established for the CATT study required the presence of leakage on fluorescein angiography as well as evidence of fluid on OCT. There was no transfer of participant information between the two centers while determining eligibility or detailed grading of baseline CNV. This enabled an unbiased masked grading in both reading centers that should be of help in correlative studies of morphometric features observed in angiography and OCT.
The masked replicate gradings that were re-assessed by the graders yielded good reproducibility results. The results of quality assurance measures of contemporaneous variability were good and in agreement with previous reports20,21. The least agreement was observed for assessments requiring clear identification of the foveal center a task that was difficult in eyes where the lesion or its components were partially present within the foveal avascular zone.
The grading protocol used in this study, in which every eye was assessed by two graders and a consensus grading was obtained for both qualitative and quantitative measurements, yielded reproducible results with excellent intra- and inter-grader agreements.
In summary, our study shows very good reproducibility of the qualitative and quantitative assessments used in the analysis of the morphometric characteristics of AMD lesions present in participants of the CATT trial. As expected, there is a strong association between baseline AMD lesion characteristics and baseline visual acuity.
The authors would like to acknowledge the following individuals for their contributions: Dr Stuart L. Fine, MD, for extensive guidance and knowledge, and Dr Robert A. Mittra, MD for his insightful review of the scientific content of this manuscript.
Financial Support: This study was supported by cooperative agreements U10 EY017823, U10 EY017825, U10 EY017826, and U10 EY017828 from the National Eye Institute, National Institutes of Health, Department of Health and Human Services. ClinicalTrials.gov number, NCT00593450
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Conflict of Interest: None
The author(s) have made the following disclosure(s): Juan E Grunwald: none; Ebenezer Daniel: none; Gui-shuang Ying: none; Maxwell Pistilli: none; Maureen G. Maguire: none; Judith Alexander: none; Revell Whittock-Martin: none; Candace Parker: none; Krista Sepielli: none; Barbara A. Blodi: none; Daniel F. Martin: none.
Institutional Review Board Approval The project was conducted in accordance with the principles of the Declaration of Helsinki, with the approval of the governing Institutional Review Boards of each institution. This study complied with the Health Insurance Portability and Accountability Act regulations.