|Home | About | Journals | Submit | Contact Us | Français|
Software-based image analysis is important for studies of cartilage changes in knee osteoarthritis (OA). This study describes an evaluation of a semi-automated cartilage segmentation software tool capable of quantifying paired images for potential use in longitudinal studies of knee OA. We describe the methodology behind the analysis and demonstrate its use by determination of test–retest analysis precision of duplicate knee magnetic resonance imaging (MRI) data sets.
Test–retest knee MR images of 12 subjects with a range of knee health were evaluated from the Osteoarthritis Initiative (OAI) pilot MR study. Each subject was removed from the magnet between the two scans. The 3D DESS (sagittal, 0.456 mm×0.365 mm, 0.7 mm slice thickness, TR 16.5 ms, TE 4.7 ms) images were obtained on a 3-T Siemens Trio MR system with a USA Instruments quadrature transmit–receive extremity coil. Segmentation of one 3D-image series was first performed and then the corresponding retest series was segmented by viewing both image series concurrently in two adjacent windows. After manual registration of the series, the first segmentation cartilage outline served as an initial estimate for the second segmentation. We evaluated morphometric measures of the bone and cartilage surface area (tAB and AC), cartilage volume (VC), and mean thickness (ThC.me) for medial/lateral tibia (MT/LT), total femur (F) and patella (P). Test–retest reproducibility was assessed using the root-mean square coefficient of variation (RMS CV%).
For the paired analyses, RMS CV % ranged from 0.9% to 1.2% for VC, from 0.3% to 0.7% for AC, from 0.6% to 2.7% for tAB and 0.8% to 1.5% for ThC.me.
Paired image analysis improved the measurement precision of cartilage segmentation. Our results are in agreement with other publications supporting the use of paired analysis for longitudinal studies of knee OA.
Osteoarthritis (OA) of the knee is a prevalent disease with a large social and economic cost [1, 2]. Due to the increased lifespan of the population and shifting demographics, the number of individuals with OA is increasing. A 2003 study estimated the cost of arthritis to the United States economy to be over $116 billion in 1997 dollars  and the economic problem is of similar magnitude in European countries . New drugs and treatments for OA require evaluation by methods that best reflect disease progression. Radiographic measures of joint space provide only an indirect measurement of cartilage loss and do not incorporate other soft tissue changes such as those in the menisci, and show significant loss only when joint damage is already present [3, 4]. MRI offers a more precise tool for assessing cartilage degradation [5, 6], since cartilage can be visualized directly.
MRI data sets can be assessed visually using qualitative or semi-quantitative methods whereby a reader evaluates the image features according to a consistent set of rules or by comparison to representative cases from an atlas [3, 7-9]. MRI data sets are inherently digital, therefore, it is also possible to use software-based methods to evaluate the images and produce outcome measures for clinical studies. A common class of these software applications is the segmentation algorithm. The goal of segmentation software is to outline the relevant structure in space and identify which pixels or voxels belong to the object. For some clinical applications it is often critical to characterize the shape of the segmented structure to aid in diagnosis. In mammography, for example, analysis software is also employed to determine the level of spiculation for a lesion . For knee OA assessment, the goal of the segmentation software is to outline the articular cartilage in three dimensions and calculate the volume, thickness, and surface area or other biomarkers. There are different approaches and software tools published in the literature to achieve 3D segmentation of the articular cartilage and this is an active area of research [5, 11-16].
A “fully automated” application would accurately delineate the cartilage on each slice in the data set without the need for any reader or observer to guide the process. Such an automated software tool could be considered the ultimate goal of these endeavors; however such a tool is not currently available. Although there has been some effort at creating fully automated segmentation software , normal anatomical variation and different degrees of disease status produce a wide range of image appearance for which the application of a single software approach is unlikely to be 100% successful. Therefore, a “semi-automated” method, which uses a skilled reader in combination with image-analysis software, may offer the most practical approach to cartilage segmentation.
We previously published a validation study of a semi-automated software tool which uses a hybrid approach  similar to other methods . This tool can measure the cartilage volume (VC), cartilage and bone surface area (AC and tAB), as well as mean and minimum thickness (ThC. me and ThC.min). Additional imaging-based biomarkers can also be computed from the segmented images . With semi-automated analysis, an expert reader with anatomic knowledge is required to guide the software. The software has numerous automated image-processing tools to increase the speed and precision of the cartilage segmentation. We present here the results of a second component of software validation. The initial validation study investigated the re-measurement reproducibility of cartilage volume and thickness when analyzed in a completely blinded manner . While the software tool as reported  could be used to evaluate longitudinal studies of knee MRI, it has been previously shown that paired image analysis will reduce the measurement precision errors and this approach has become the standard approach used for longitudinal clinical trials . We hypothesized that a semi-automated software method, which permitted the reader to simultaneously evaluate images series from multiple time points of a study, would increase the precision of the segmentation. To this end, we have modified the existing software tool to permit a reading of paired data sets. In this study, we present a validation of the method using cross-sectional test–retest knee MRI data.
Our study used images from the pilot MR study for the Osteoarthritis Initiative (OAI). The OAI is jointly sponsored by the National Institute of Arthritis and Musculoskeletal and Skin Diseases (NIAMS), several other Institutes at the National Institutes of Health (NIH), and the pharmaceutical industry. The OAI enrolled 4,794 subjects and will acquire MRI exams of both knees on every subject at baseline as well as at a minimum of four annual follow-up exams, providing nearly 48,000 individual knee exams. In addition to the OAI, there are other large-cohort studies and clinical trials using knee MRI that will require fast and low-cost tools to evaluate the substantial number of subjects [20-22]. The cost of a study can be reduced by using a faster method that is equally accurate, reproducible, and sensitive to longitudinal change as manual assessments since a major expense of these studies is quantitative evaluation.
Data and images were obtained as part of the OAI pilot MR study. The pilot study was performed at the Ohio State University and Memorial Hospital of Rhode Island under review and approval of the local institutional review boards. MRI exams were performed on 12 knees from ten adult subjects (three men and seven women). The participants were selected according to the OAI study design  and were participants in other OAI pilot MR studies [15, 19, 24, 25]. All ten subjects met the requirements for the incident cohort of the OAI. The six knees (five subjects) classified as having symptomatic knee OA had pain, aching, or stiffness on the majority of days within a month in the past 12 months, and had OA diagnosed by a physician. Another three knees (three subjects) had only infrequent knee pain, aching, or stiffness over the prior year. Seven of the ten subjects in this pilot study also participated in the OAI (incidence cohort) and underwent fixed-flexion posterior-anterior (P/A) radiography  of both knees. The last three knees (two subjects) were asymptomatic and had no prior diagnosis of OA and did not participate in the OAI. Based on the OAI knee radiographs, five subjects had Kellgren–Lawrence grade (KLG) 1, one subject had KLG 2, and one subject had KLG 3. The mean age of the participants was 52.2 years (range, 45–73 years) and the mean body mass index (BMI) 28.2 kg/m2 (range: 21.8–34.6 kg/m2). Each subject was imaged twice to permit measurement of repositioning reproducibility. The subjects were removed from the scanner and table, allowed to walk for about 10 min, and were then repositioned and reimaged.
The images were obtained with a Siemens (Erlangen, Germany) Trio 3-Tesla (3T) MR system and a USA Instruments (Aurora, OH) quadrature transmit/receive extremity coil using a 3D sagittal dual echo in the steady state (DESS) acquisition with water excitation. The slice thickness was 0.7 mm, the in-plane pixel size was 0.365 mm×0.456 mm, TR was 16.5 ms, TE was 4.7 ms, and each acquisition produced 160 images.
The semi-automated software initially reported performed cartilage segmentation on a single knee 3D MRI acquisition  using an “active contour model” algorithm  to refine the segmented margin on each slice of the 3D image series. The active contour model step serves to provide amoreprecise and objectivesegmentationbymaking automatic modifications to the computer-determined cartilage margins until the full contour best matched the true anatomical edge. The steps are demonstrated in Fig. 1 and below. Segmentation of each successive slice is performed by copying the final segmentation from the previous slice and applying the active contour algorithm. Manual correction of mis-identified cartilage boundaries can be performed at each step. The manual correction was done by two readers with more than 3 years experience of cartilage segmentation.
With the new image-analysis method, a matched pair of image sets is analyzed using the previously described technique with some modifications made to improve speed, convenience, and objectivity. The reader first performs a complete segmentation of one 3D image series using the method described above. Once completed, the software then displays a single slice from each acquisition of the pair. A tool is provided that allows the user to scroll through the images from the second 3D acquisition until the displayed images for both scans best match (one dimensional image alignment, without slice interpolation). From this point forward, the image series are synchronized so the reader can scroll through both scans simultaneously. Assuming identical slice thickness, scan prescription, and knee positioning, the two scans will continue to match as the reader scrolls though the images. In practice, minor adjustments are made by the reader during this process to provide the best agreement between the pair of images. Segmentation of the second scan is then accomplished by first copying the correct segmentation from the matched slice of the initial image set and then applying the active contour model. As above, the software tool then applies the initial segmentation to each successive slice, the active contour algorithm performs the correction, and the reader adjusts or corrects any anomalous boundaries.
After both knees have been segmented, the software counts the number of pixels contained within each contour, multiplies by the pixel size, and sums the results from all slices. This value represents the volume of articular cartilage (VC). The bone surface area (tAB) is the cartilage boundary adjacent to the bone; the cartilage surface area (AC) is the opposite cartilage boundary. The cartilage thickness (ThC) is computed by the distance perpendicular from the bone surface to the cartilage surface. After these computations were performed, the root-mean square coefficient of variation (RMS CV%) was used as a metric to quantify the test–retest reproducibility .
For all segmentations, the paired analyses RMS CV% ranged from 0.9% to 1.2% for VC, from 0.3% to 0.7% for AC, from 0.6% to 2.7% for tAB and 0.8% to 1.5% for ThC. me (Table 1).
We also examined the reproducibility for the knees with clinical diagnosis of OA and those from the other volunteers independently. For knees with clinical OA, the RMS CV% ranged from 0.7% to 1.2% for VC, from 0.2% to 1.0% for AC, from 0.7% to 2.8% for tAB and 0.7% to 1.8% for ThC.me (Table 2). For the other knees, the RMS CV% ranged from 0.8% to 1.3% for VC, from 0.3% to 0.6% for AC, from 0.6% to 2.6% for tAB and 0.8% to 1.2% for ThC.me (Table 3). There was no significant difference in reproducibility between the two groups which indicates that the presence of osteophytes and thinning cartilage should not be problematic for analysis.
The goal of our study was to evaluate a software tool to segment test–retest 3D MRI series in a pair-wise manner for use in longitudinal studies of knee OA. We found excellent reproducibility for all cartilage plates. In addition, no substantial difference between the knees with clinical OA and the other knees was found which suggests that the method is not influenced by obscured or ambiguous cartilage margins, or osteophytes that are often present in OA. Our re-measurement reproducibility results compare favorably with other published studies using a paired-reading method [24, 29-32]. Eckstein et al. performed a similar paired analysis of VC, AC, and tAB of the MT and LT using the identical data set . The RMS CV% values are similar and ranged from 1.2% to 2.2%  compared to 0.5% to 2.7% for this study.
As with unpaired image segmentation , the paired analysis software uses a semi-automated active contour algorithm to trace the cartilage borders. For segmentation of a single 3D image series, the active contour is applied to successive slices in a scan, while for the paired readings, the active contour is applied to matched-slices from different exams of the same subject. For both unpaired and paired image analysis, the semi-automated software algorithm is fast and robust since each segmentation begins with an accurate initial estimate. The active contour model also performs the segmentation according to a mathematically rigorous algorithm, and minimizes the amount of reader correction and hence the impact of their judgment, which improves the reproducibility.
In a previous reproducibility study of similar data using unpaired analysis and our segmentation software, the intrareader reproducibility was considerably higher than the results reported here . This is expected since prior work has shown unpaired precision to be much larger than that of paired segmentation analysis [29, 33]. We have also found reader judgement to be a dominant factor for cartilage segmentation reproducibility and that substantial improvement is possible once this factor is mitigated. Improved reader reproducibility could be accomplished by additional training as well as by increasing the level of software automation. To the extent that the new software is more automated, the results will be less influenced by the reader, and the reproducibility should improve. Although the sensitivity of our segmentation algorithm has not yet been tested on longitudinal MRI exams, the work presented in this report is the first step to validate the methodology. Once the rate of longitudinal cartilage change has been quantified, the results presented here can be used for power calculations for designing clinical studies of OA using MRI.
In longitudinal studies of knee OA, the active contour refinement step should produce a segmentation that reflects any local cartilage changes between scans since it is designed with sufficient flexibility to vary the results and conform to the true bone and cartilage contours. Since the majority of the cartilage morphometry is likely to remain stable, using a paired segmentation as a starting point is an efficient method to segment each slice. Measures derived from a total cartilage plate or even a subregion may not reflect progression precisely since small changes are likely to be insignificant compared to the error associated with the measurement . Therefore, quantifying changes for each defect individually may prove to the most favourable technique to observe progression. In future longitudinal studies, we anticipate using 3D image registration (spatial matching of baseline and follow-up exams) to examine changes in the region adjacent to a cartilage defect.
There was evidence of systematic bias in several of the morphometry measures when comparing the baseline to follow-up values. As an example, we show Fig. 2, a Bland Altman plot for the cartilage surface area (AC). This systematic change in the measurement demonstrates a potential shortcoming of the method, which suggests further improvement to the re-measurement reproducibility may be possible once the source of the bias is understood and corrected.
In agreement with prior studies, we found paired analysis improves the segmentation reproducibility compared to separate independent (unpaired) analysis. In most instances, reader judgement error decreases the reproducibility, especially in cases with cartilage defects and other abnormalities found over the course of OA . Using paired analysis the reader can directly visualize the previous findings and the error can be reduced resulting from judgement of cartilage borders, by making a more consistent evaluation.
In contrast with most prior studies, we found no difference in reproducibility between the knees with and without clinically defined OA [15, 29, 32]. This is anticipated because the paired analysis technique allows the reader can directly visualize the previous findings and systematically select appropriate contours to accommodate osteophytes growth.
One limitation of this study is the relatively small number of subjects; it is difficult to draw definite conclusions since even a single case can skew the results. A further limitation is the use of only one scanner type and pulse sequence, but is not anticipated to be problematic or to require changes to the approach or to the software. However, the applicability of the analysis software to different MR system vendors and pulse sequences should also be evaluated. We have tested the cartilage segmentation technique using high quality data, from a well standardized study, with thin slices and high in-plane spatial resolution. All cartilage segmentation is potentially less precise on lower quality data as has been shown in other studies [30, 35]. Another limitation is that the results from this study were not validated using histological or arthroscopy as this was a pilot study with volunteers and no surgical intervention was included. Since each reader evaluated a separate portion of the data set, we were not able to measure the inter-reader reproducibility. In this study, we did not examine cartilage subregions, which have been shown to be more sensitive to change than the total cartilage plates 
The next step in validation of this paired analysis software approach is to perform measurements OA progression rates on longitudinal image data with the reader blinded to time point. Additional development of the core image-processing algorithms should enable further increase of the level of automation, thereby producing a quicker, more objective and robust tool which is free of systematic bias. Furthermore, we plan to add features to the graphical user interface to permit segmentation of more than two visits of a longitudinal study and to implement 3D image registration to track individual cartilage lesions or defects.
Software algorithms and user interfaces allow for fast and reproducible analysis of paired scans for articular cartilage segmentation. Such methods are necessary tools for use in large longitudinal studies of knee OA and are generally applicable to analyze other anatomic features. In addition to VC, AC, tAB, and ThC, it is also possible to measure other image-based metrics using segmented MRI data sets such as those described in Eckstein et al. 
The Osteoarthritis Initiative (OAI) and this pilot study are conducted and supported by the National Institute of Arthritis and Musculoskeletal and Skin Diseases in collaboration with the OAI Investigators and Consultants. This manuscript has been reviewed by the OAI Publications committee for scientific content and data interpretation. The research reported in this article was supported in part by contracts N01-AR-2-2261, N01-AR-2-2262 and N01-AR-2-2258 from NIAMS. Support for this project was also provided by a contract with the NIAMS intramural program.
This work was also supported by a contract with the NIH/NIAMS intramural program. We would like to thank Raphaela Goldbach-Mansky of the NIH/NIAMS Intramural Research Program for her support in developing early versions of the software.
NIAMS funded this work in part (contracts N01-AR-2-2261, N01-AR-2-2262 and N01-AR-2-2258).
Conflict of Interest Statement ES, RJ, JY, and CBE received direct salary support or had fee for service contracts associated with the OAI. In particular:
ES is the principal of SciTrials, LLC, is the NIAMS OAI Technical Advisor and is under contract to NIAMS for this purpose; RJ and JU are at The Ohio State University that is under contract (N01-AR-2-2261) as a clinical center for the OAI; CBE is at the Memorial Hospital of Rhode Island that is under contract (N01-AR-2-2262) as a clinical center for the OAI.