The LINA toolkit has been applied to lymphoma patient follow-up studies in the experiments. Patients included 6 men and 3 women. The median age was 29 (range 20–63). The total number of lesions identified on imaging was 34 and the mean number of lesions per patient was 3.7 (range, 1–7 lesions). In the first experiment, we evaluated the accuracy of the longitudinal registration algorithm using chest lung CT images, and in the second experiment, the quantitative follow-up results of the proposed method are compared with the manual results.
Quantitative evaluation of the algorithm
The proposed longitudinal deformable registration algorithm has been evaluated by using randomly simulated lung CT images. First, for each series of the sample images, the demon’s algorithm (45
) was used to register the subsequent images onto the first time-point image. Then, one sample image was selected as the template image, and all the other first time-point images of other subjects were deformed onto the template space using the same deformable registration algorithm. Finally the statistical model for the spatially warped/normalized temporal deformations were calculated, which was then used to randomly generate the longitudinal deformation of the template image. In this paper, the statistical model of deformation (SMD) presented in (46
) was used to train such a model. In this experiment we used the chest CT images obtained from The Methodist Hospital as our datasets and the resolution is the same as our patient data. One image was selected as the template and other 19 images were selected as the training samples, and ten CT images were randomly simulated. In a similar way, the longitudinal changes of these CT images can then be simulated and applied to the ten CT images (four serial images each). The registration errors with the ground truth were calculated reflecting the accuracy of the registration:
=10 is the number of testing images, and |Ω| represents the number of voxels in the template image domain. fi,v
is the resultant deformation at voxel v
for testing image i
is the corresponding ground truth from the simulated deformation fields.
In order to evaluate the registration accuracy, we compared the results using the LINA algorithm and the FFD-based registration (used in (25
)). The distribution of the registration errors for the LINA algorithm and the FFD-based registration algorithm are plotted in . The mean registration error for LINA was 3.3mm
, and the mean registration error for FFD-based registration was 4.6mm
. The results showed that the registration errors were found in less than the largest voxel spacing of the simulated images. Compared to the voxel size of the PET image, 5.47mm
Therefore we can achieve sub-voxel average registration accuracy, which is accurate enough to calculate quantitative measures from PET images. As for the segmentation error, the average overlapping between the automatically segmented ROIs and the manually marked ROIs are ~85%, indicating relatively high accuracy in terms of ROI segmentation and automatic mapping.
Comparison of the registration errors between LINA and FFD-based registration using simulated images.
In the second experiment we compared the proposed automatic ROI mapping based on longitudinal registration with the semi-automatic/manual segmentation methods for ROI. First, the FLIRT program (47
) was used to co-register the PET/CT images at same time-point. For each CT image, a radiologist delineated all the ROIs using our semi-automatic segmentation tools. In the automatic ROI mapping method, the segmented ROIs from the baseline were automatically mapped onto all the CT images in the follow-up time-points. Finally, the SUVs of each ROI were calculated in the original PET image space by transforming the corresponding regions to PET, so that no interpolation of the PET images is performed.
For a dataset with PET (image size 128×128×300) and CT (image size 512×512×300), the average time for co-registration was about 8 minutes on HP xw4400 workstation by using the FLIRT program. The time for deformable registration of two CT images was dependent on the sub-volume to be registered. For sub-volumes cut from the CT with size 256×256×70, the calculation time is less than 200 seconds.
We compared the results of automatic ROI mapping with the original semi-automatic/manual segmentation results, using SUV values of each region. The mean of the relative squared differences between two SUVmax values for all the regions of all the patients are calculated as follows,
denote the segmentation results using the semi-automatic/manual segmentation and the proposed automatic mapping for the i
th, respectively. N
is the total number of ROIs under evaluation. For all the ROIs of the nine patients we studied, this average normalized squared difference is 0.02. Therefore, the computer-assisted quantitative analysis method obtains similar results with the “gold standard.”
Segmentation and visualization of longitudinal ROI mapping
First, the semi-automatic and manual ROI segmentation are illustrated in and , respectively. From we can visually identify the region of interest, and then manual points are initialized inside the lymph nodes (), and the algorithm automatically extracts the boundaries of the lymph nodes as shown in . When the boundaries of an ROI are blurry, e.g., lymph nodes are connected with muscles, the level set algorithm normally leaks into a larger region, and a manual segmentation of the ROI is then applied. From we identified a region of interest but from the initial point shown in , the level set segmentation leaks into a larger region in the CT image. Therefore manual segmentation of the ROI is necessary. shows the manually marked region.
Fig. 3 Illustration of semi-automatic lymph node segmentation using level set method. (a) The region of interest is first identified by overlapping the PET with the CT; (b) an initial point is then manually marked for each lymph node from CT image; and (c) the (more ...)
Fig. 4 An illustration of manual segmentation of ROI where boundaries are not clearly discernable, in which the semi-automatic method has failed. (a) The ROI is first identified by overlapping the PET with the CT; (b) the level set segmentation leaks into a (more ...)
It is worth noting that the PET signal can also be used to facilitate the segmentation process. For example the level set evolution can stop to grow when the PET signal is weak. However, since the PET signal is the one that we need to measure, it will introduce biased quantitative calculation if the selection of ROI is dependent on the PET signal itself. Thus in this study we used the strategy that the ROIs are only selected from the CT images.
Based on the proposed algorithm, a longitudinal imaging navigation and analysis tool (LINA) is developed. The key functions include a sophisticated data management system to facilitate quantitative longitudinal studies and statistical analysis; an exceptional 4-D Viewer of longitudinal images with underlying longitudinal deformations and 4-D ROIs; and the quantitative results for the follow-up study of each patient. In this section, we introduce the visualization of some results. Compared to the radiological reports, the new tool will provide a platform for comprehensive dataset management.
shows an example of the results with two time-points, and the lesion is in the lung. show the fused PET/CT images at the baseline and the second time-point, respectively. show the semi-automatic segmentation of the lesion, respectively. After deformable registration, the baseline image can be warped onto the image at second time-point, and shows the registered result, in which the green curve illustrates the automatically transformed ROI from the baseline CT. Comparing and , we can see that after warping the baseline image can be very similar with the second time-point image, and the shape and volume of the transformed ROI is also very similar to the semi-automatically determined ROI at the second time-point. Regarding SUVmax values at the second time-point, the semi-automatic results for the lesion and liver are 2.04 and 1.81, separately, and the SUVs of the automatic mapping results at second time-point are the same. This is because the manual and automatically mapped ROIs are highly overlapping. The SUVmax consistency of automatic and semi-automatic segmentation results indicates that the automatic segmentation method is effective for SUV calculation of ROI. In addition, QI of the ROI at the two time-points are 2.03 and 1.13 respectively, which indicate the treatment outcome of lymphoma. This decrease demonstrates that treatment had a good effect on lymphoma.
Fig. 5 Visual comparison of the semi-automatic segmentation of lesion and the automatically mapped lesion. (a) and (b) fused PET/CT images at the baseline and the second time-point, respectively; (c) and (d) semi-automatic segmentation of lesion; (e) warped (more ...)
shows another example with four time-points, where lymph nodes can be clearly segmented. The first row shows the fused PET/CT images, where lymphoma region has obvious high intensity. The CT images are shown in the second row, and the yellow curves illustrate the semi-automatic lymph node segmentation results. For the follow-up CT images, we applied the serial image registration algorithm and warped the baseline image onto each subsequent CT image shown in row 3. Correspondingly, the ROI at baseline can be transformed onto these CT images, shown as the green contours in row 3. At each follow-up time-point, the warped baseline image appears similar to the CT image captured at that time-point. By using the automatically mapped ROIs, SUVmax and QI measurements can be automatically generated for each case. These data can be used to quantitatively assess metabolic activity in specific lesions over multiple imaging datasets which is critical for evaluating therapy response. From the QI values given, we can see the positive response of the treatment quantitatively in this case.
Fig. 6 Comparison of the semi-automatic segmentation of lymph node and the automatically mapped shapes. Row 1: fused PET/CT images at different time-points; Row 2: semi-automatically segmented lymph nodes; Row 3: the warped baseline CT image and the corresponding (more ...)
After calculation of QI (the ratio of maximal SUV of lesion to the mean SUV of the liver), they can be plotted longitudinally and supplies as quantitative response of treatment. shows some results in our datasets. Each figure shows the plots of QIs of all ROIs of one patient at different time-points. For the same longitudinally corresponding ROI, we used lines with the same color to link them, and solid lines show the semi-automatic/manual results and dashed lines indicate the results of the automatic mapping results. The big solid circle markers indicate that semi-automatic/manual segmentations at that time-point are automatically mapped onto other time-points for automatic SUV calculation.
Fig. 7 Plots of longitudinal QI values of different patients for both of semi-automatic segmentation method (dash-dot lines) and the proposed automatic mapping method (solid lines). (a) Seven lesions, four time-points; (b) one lesion, three time-points; (c) (more ...)
show two examples where semi-automatic segmentation of ROI is performed from the baseline. By comparing the solid lines with the dashed lines, it can be seen that the longitudinal QI values obtained by using both methods are in reasonable agreement. The longitudinal change of QI of each ROI indicates that these ROIs are obvious response to treatment within the first five and half months, while the situation might become worse between 5.5 to 8.5 months. In the computing results of another patient with one ROI at three time-points are shown. Similarly, we see very close measure between the two methods, and the longitudinal plots of the QI values suggest a positive response of the treatment.
show two examples where it was difficult to identify ROIs in some images. shows the results of one patient with four ROIs at five time-points, in which it is difficult to identify the ROIs at the fifth time-point except for one. All the lesions are determined from the baseline image and the computer-assisted method can determine the corresponding lesions for all the follow-up studies, while it is difficult to even manually mark the lesion for some scans. The results show that our method can delineate other three ROIs at the fifth time-point by using serial CT registration, which assists oncologist to quantitatively evaluate the treatment response. In , two ROIs can be detected at the first time-point (blue and red point markers), and the other two ROIs indicated by green and magenta markers can be identified at the second time-point. By using the proposed computer-assisted quantitative approach, we can automatically calculate the SUV values of along the image series and hence prove a comprehensive quantitative evaluation of the treatment outcomes of lymphoma. indicate that our proposed method enable to provide pro- and post-perspective analysis for treatment outcomes of lymphoma.
From the experimental results, one can see clearly that this method has the following advantages. Since lymphomas are often gathered together in some patients, it is difficult to map one-to-one correspondence among the lymphoma regions segmented by hand at different time-points. Our method is able to automatically segmented ROIs from follow-up CT images by using ROI mapping based on the longitudinal correspondences, and it is more efficient to accomplish the goal of quantitative analysis; the novel analysis tool enables us to show the longitudinal image data with ROI in 4-D. The volume and QI of ROIs can be visualized in time sequence. The plots of QI supplies a quantitative evaluation means for treatment response of lymphoma. All these functions simplify the analysis and improve the efficiency of analysis.
It is impractical to only show the longitudinal curves like to draw conclusions of the therapeutic responses. Our rationale is that by using the LINA tool, users/experts not only visualize the lesions from the aligned CT images and overlap PET images onto the CT, but also visualize the quantitative values (such as SUVmax, QI) for selected lesions. Thus it would be more efficient and informative to conduct quantitative imaging for follow-up study of lymphoma patients by combining both of ROIs in longitudinal images and corresponding longitudinal PET values.
It is worth noting that computer–assisted analysis method might occasionally cause computation errors. In our experiments, the only one error occurred when the lesion size is small (<12 mm). There are several factors affecting this error. First, the precision of PET/CT co-registration will undoubtedly affect the accuracy of SUV calculation of lymphoma region especially when the lesion is small and does not have clear boundary from CT images. This was the case why the error occurred. Another factor is that whether the automatically mapped ROIs are accurate enough to cover the PET peak region. Based on our validation of the algorithms, the matching accuracy for longitudinal CT series is about 3.3mm for resolution 0.98mm×0.98mm×3.75mm, and it is less than one voxel size. Therefore, in the case where lesion size is small (<12mm), we applied a relatively large ROI to cover the whole PET peak spot, i.e., grow the accurate ROI by 3.75mm in all directions. In this way, all the 34 lesions can be successfully analyzed using our computer-assisted quantitative analysis tool.