|Home | About | Journals | Submit | Contact Us | Français|
The objective of this study was to compare the sensitivity of detection of lung nodules on low-dose screening CT images between radiologists and technologists.
11 radiologists and 10 technologists read the low-dose screening CT images of 78 subjects. On images with a slice thickness of 5 mm, there were 60 lung nodules that were ≥5 mm in diameter: 26 nodules with pure ground-glass opacity (GGO), 7 nodules with mixed ground-glass opacity (GGO with a solid component) and 27 solid nodules. On images with a slice thickness of 2 mm, 69 lung nodules were ≥5 mm in diameter: 35 pure GGOs, 7 mixed GGOs and 27 solid nodules. The 21 observers read screening CT images of 5-mm slice thickness at first; then, 6 months later, they read screening CT images of 2-mm slice thickness from the 78 subjects.
The differences in the mean sensitivities of detection of the pure GGOs, mixed GGOs and solid nodules between radiologists and technologists were not statistically significant, except for the case of solid nodules; the p-values of the differences for pure GGOs, mixed GGOs and solid nodules on the CT images with 5-mm slice thickness were 0.095, 0.461 and 0.005, respectively, and the corresponding p-values on CT images of 2-mm slice thickness were 0.971, 0.722 and 0.0037, respectively.
Well-trained technologists may contribute to the detection of pure and mixed GGOs ≥5 mm in diameter on low-dose screening CT images.
The National Lung Screening Trial, a randomised controlled trial comparing low-dose CT and chest radiograph for lung cancer screening, has demonstrated 20% mortality reduction in lung cancer patients following low-dose CT lung cancer screening, compared with chest radiographic screening . In Japan, 60% of medical check-up facilities provide the option of low-dose CT lung cancer screening to potential screening candidates. With advances in CT technology, multislice CT has been developed, and more and more advanced techniques are evolving rapidly. As a result, an enormous number of CT images are being produced every day and the workload of radiologists is increasing. Double reading of chest CT by radiologists does not always work efficiently . Although systems for computer-aided diagnosis (CAD) are being developed, such systems are not always available in daily practice and are not applicable to low-dose CT lung cancer screening at present. One of the obstacles in the introduction of low-dose CT lung cancer screening by multislice CT is the increasing burden it places on radiologists in terms of reading an enormous number of CT images. In the field of screening mammography and CT colonography (CTC), several investigators have reported possible useful roles for radiological technologists in reading of mammograms and CTC images [3-10]. However, to date, no study has been reported on the possible roles of radiological technologists for the detection of lung nodules in low-dose CT lung cancer screening.
The purpose of this study was to compare the sensitivity of detection of lung nodules between radiologists and radiological technologists in low-dose CT lung cancer screening.
This study was conducted with the approval of the institutional review board and informed consent was obtained from each of the screenees.
Low-dose screening CT images of 78 subjects (34 females and 44 males; mean age, 64 years; range, 46–78 years) were used for this study. Of the 78 subjects, 48 had lung nodules ≥5 mm in diameter, while 30 did not haveany lung nodules. The screenees in our low-dose screening CT project were self-referred. The 78 subjects were not consecutive screenees but were preferentially selected for this study. The CT scanner used for this study was the Aquilion 16 scanner (Toshiba Medical Systems, Otawara, Japan). The scanning conditions for low-dose CT lung cancer screening were tube potential 120 kVp, current 30 mA, collimation 1 mm×16 rows, 0.5 s rotation−1, pitch factor 0.69, field of view (FOV) 32 cm and FC01 kernel. CT images with a slice thickness of 2 mm were reconstructed at 2-mm intervals and those with a slice thickness of 5 mm were reconstructed at 5-mm intervals from the same raw data acquired on the same day. Since CT screening for lung cancer in Japan is currently performed by low-dose thick-section CT (TS-CT slice thickness, 5–10 mm) as well as low-dose thin-section CT (slice thickness, ≤3 mm), we used 2-mm sections and 5-mm sections as representative thicknesses in each group. The consistency of lung nodules was classified into three types using thin-section CT (TS-CT): nodules with pure ground-glass opacities (GGOs; non-solid nodules); nodules with mixed GGOs (GGOs with a solid component; part-solid nodules); and solid nodules [11-13]. TS-CT of the nodules was performed with a scan range of no more than 4 cm under the following conditions: tube potential 120 kV, current 300 mA, beam collimation 0.5 mm×16 rows, pitch factor 0.69 and 0.5 s rotation−1. The TS-CT images were reconstructed at 1-mm intervals, a slice thickness of 1 mm, an FOV of 22 cm and super-high-resolution sharp lung algorithm (FC82). On CT images with a slice thickness of 5 mm, the number of the lung nodules that were ≥5 mm in diameter was 60; these nodules were visualised as pure GGOs in 26 cases, as mixed GGOs in 7 cases and as solid nodules in 27 cases on TS-CT images (12 out of the 48 cases had at least two nodules that measured ≥5 mm in diameter on the images with a slice thickness of 5 mm). On CT images with a slice thickness of 2 mm, the number of the lung nodules that were ≥5 mm in diameter was 69; these nodules were visualised as pure GGOs in 35 cases, mixed GGOs in 7 cases and solid nodules in 27 cases on TS-CT images (20 out of the 48 cases had at least two nodules that measured ≥5 mm in diameter on the images with a slice thickness of 2 mm). These nodules were detected by double reading (one radiologist had 6 years' experience and the other had 24 years' experience in reading chest CT) of the low-dose screening CT images. In addition, these nodules were reconfirmed as the gold standard by another radiologist who had 26 years' experience in reading chest CT. The radiologist with 24 years' experience in reading chest CT measured the maximal diameters of the lung nodules on CT images with both the 5-mm and 2-mm slice thickness, and classified the consistency of the lung nodules on TS-CT.
11 radiologists and 10 radiological technologists participated in this study. The years of experience of the 11 radiologists in reading chest CT ranged from 4 to 20 years (mean, 12 years), and their experience in reading low-dose screening CT was 3 years, on average (range, 1–6 years). Although the technologists had about 16 years' (range, 2–30 years) experience in CT scanning, none of them had any experience in reading low-dose screening multislice CT of the chest. Therefore, the 10 technologists learned how to read low-dose screening CT images of 10-mm slice thickness using a teaching software program  for 1 month prior to reading the low-dose screening CT images of 5-mm slice thickness. Furthermore, the 10 technologists also learned how to read low-dose screening CT images of 2-mm slice thickness using a teaching software program  for 1 month prior to reading the low-dose screening CT images of 2-mm slice thickness. In addition, another teaching software program that was developed in-house, which contained three cases with solid nodules measuring <10 mm in diameter and one case without lung nodules, was provided to the 10 technologists before they were required to read the low-dose screening images of 2-mm slice thickness. The number of solid nodules measuring <10 mm in the three cases on low-dose screening CT images of 2-mm slice thickness ranged from 17 to 23. Standard-dose CT of the other case without lung nodules was reconstructed at 1-mm intervals and 1-mm slice thickness.
The reading system consisted of four personal computers with two monitors each (a 1.3-megapixel liquid crystal monochrome display monitor for data input, and a 9 megapixel liquid crystal monochrome display monitor for viewing and measuring), and a server. 11 radiologists were asked to input the x, y and z co-ordinates of each nodule, to input the confidence level, as a percentage, of the localisation of the lung nodules, and to measure the maximal and perpendicular diameters of each nodule. They were asked to attempt to classify the consistency of the lung nodules and to differentiate between benign and malignant nodules on the 5-mm slice thickness CT images as well as the 2-mm slice thickness CT images. The reading procedures by the radiologists were performed based on the assumption that they were reporting readings in a daily screening setting. The technologists were asked to input the x, y and z co-ordinates of each nodule, and to input the confidence level, as a percentage, of the localisation of the lung nodules. The reading procedures by the technologists focused only on detection of lung nodules. In regard to the size of the lung nodules, although the target size of the nodules was ≥5 mm, the technologists were asked to click on all the nodules that they had detected, even if the size of the nodule was <3 mm, because they did not measure the nodule sizes to reduce the overall reading time, while the radiologists were asked to click on any nodules that they had judged to be ≥3 mm in diameter. The observers were informed that any confidence level would be counted. The 21 observers read the screening CT images of 5-mm slice thickness of the 78 cases at first; then, 6 months later, the same observers read the screening CT images of 2-mm slice thickness of the 78 cases. Each reading session lasted for a period of 2 days. The low-dose screening CT images were presented to the observers in random order. The reading time for each data set was recorded by the reading system.
Statistical analyses were performed using JMP v. 9 (SAS Institute, Cary, NC) and Excel 2004 (Microsoft, Redmond, WA) computer software. Lung nodules that matched the gold standard were defined as positive. Mann–Whitney's U-test was used for evaluation of the sensitivities, confidence levels, false positives, specificities and reading times. p-values of <0.05 were set as denoting statistical significance.
The results are shown in Figure 1. The mean sensitivity of detection of pure GGOs ≥5 mm in diameter was 69.6±12.9% for the radiologists [95% confidence interval (CI), 60.8–78.3%], while it was 60.4±10.6% for the technologists (95% CI, 52.8–67.9%); the difference in the mean sensitivity between the two groups was not statistically significant (p=0.095). The mean sensitivity for the detection of mixed GGOs ≥5 mm in diameter was 87.0±13.4% for the radiologists (95% CI, 77.9–96%), while it was 79.9±19.3% for the technologists (95% CI, 66.1–93.8%); the difference in the mean sensitivity between the two groups was not statistically significant (p=0.461).
The results are shown in Figure 2. The mean sensitivity of detection of pure GGOs measuring ≥5 mm in diameter was 65.5±8.8% for the radiologists (95% CI, 59.4–71.4%), while it was 63.7±10.3% for the technologists (95% CI, 56.3–71.1%); the difference in the mean sensitivity between the two groups was not statistically significant (p=0.971). The mean sensitivity of detection of mixed GGOs ≥5 mm in diameter was 85.7±9.0% for the radiologists (95% CI, 79.6–91.7%), while it was 84.3±10.5% for the technologists (95% CI, 76.7–91.8%); the difference in the mean sensitivity between the two groups was not statistically significant (p=0.722).
The sensitivity of solid nodule detection according to size and CT slice thickness is shown in Table 1. The difference in the mean sensitivity of detection of solid nodules ≥5 mm in diameter between the radiologists and technologists was statistically significant in both the CT images acquired at 5 and 2-mm slice thicknesses (p=0.005 and p=0.0037; Figures 1 and and2).2). However, the difference in the mean detection sensitivity between the radiologists and technologists in regard to nodules ≥7 mm in diameter was not statistically significant on the CT images acquired at 5-mm slice thickness (p=0.1).
The mean confidence levels of detection of lung nodules are shown in Table 2. The difference in the mean confidence level between the two groups was statistically significant on the CT images acquired at 5-mm slice thickness (p=0.0007), but not in the CT images acquired at 2-mm slice thickness (p=0.07).
The mean false positives are shown in Table 2. The difference in the mean false positives between the two groups was not statistically significant on the CT images acquired at 5-mm slice thickness (p=0.97). Even when an outlier (i.e. 616 false positives reported by one of the technologists) was included, the difference in the mean false positives between the two groups was not statistically significant on the CT images acquired at 2-mm slice thickness (p=0.48 or p=0.73).
The mean specificities are shown in Table 2. The difference in the mean specificity determined between the two groups was not statistically significant in either the CT images acquired at 5-mm slice thickness or in those acquired at 2-mm slice thickness (p=0.42 and p=0.27).
The results are shown in Table 2. The difference in the mean reading time between the two groups was statistically significant on both CT images acquired at 5-mm and 2-mm slice thicknesses (p=0.04 and p=0.007).
To the best of our knowledge, this study is the first report of participation of radiological technologists in a reading test of low-dose CT for lung cancer screening. The size criterion of lung nodules for further work-up in CT lung cancer screening laid down by the Anti Lung Cancer Association (ALCA) is ≥5 mm . In the protocol of the Early Lung Cancer Action Project (ELCAP), solid nodules ≥5 mm in diameter and non-solid nodules ≥8 mm in diameter are considered as candidates for further work-up . The ELCAP group reported that the lung cancer rate in lung nodules measuring 5–9 mm in diameter was 5.9% (14/278), while that in lung nodules measuring <5 mm in diameter was 0% (0/384) at the 1-year follow-up . Non-solid nodules (nodule with pure GGO) and part-solid nodules (nodule with mixed GGO) are more likely to be malignant than solid nodules . Although the mean sensitivity of detection of solid nodules ≥5 mm in diameter by the technologists was significantly lower than that by the radiologists, there were no statistically significant differences between the technologists and the radiologists in the mean sensitivities of detection of pure GGOs and mixed GGOs ≥5 mm in diameter on either the 2-mm or the 5-mm slice thickness CT images. Therefore, the technologists may play a useful role in the detection of malignant nodules in low-dose CT lung cancer screening. It might be difficult for the technologists to differentiate small solid nodules from normal structures, such as the pulmonary vessels, because the attenuation and the shape of small solid nodules on CT images are almost the same as the attenuation and the shape of adjacent pulmonary vessels.
Skill mix, which is thought to be a concept originating from England , is one of the keywords in screening mammographies; several investigators have reported the useful contribution of radiologic technologists in pre-reading and double reading of mammograms . One study evaluated the effects of review of mammograms by radiological technologists in a population-based breast cancer screening programme; a substantial number of cancers were detected as a result of the technologists' review . Another study assessed the ability of technologists to accurately classify screening mammograms as either showing negative findings or requiring follow-up; technologists and radiologists agreed in 82% of the cases . Adding independent double reading of mammograms by technologists to independent double reading by radiologists was effective in detecting additional cases of breast cancer .
Recently, with the advent of multislice CT, CTC has been introduced in clinical and screening settings. Well-trained technologists showed almost the same performance in polyp detection as that of the radiologists . The authors of that study concluded that deployment of radiographers as reviewers is acceptable in CTC. Another study found that CTC interpretation can be adequately performed by inexperienced radiologists, radiology fellows and radiology technicians after they undertake a computer training program . In that study, one of the three technicians achieved a sufficient level of competence within 175 training CT colonographic examinations, with a lesion prevalence of about 50% for CT colonographic evidence of lesions measuring 6 mm or larger. Five radiographers underwent training in CTC via a tele-training program mainly based on the interpretation of 75 training cases . There was a statistically significant improvement in per-polyp sensitivity for lesions measuring ≥6 mm among the test cases. The authors concluded that the tele-training program was effective for obtaining a good performance of the radiographers in detecting tumoral lesions in CTC. In the field of low-dose CT lung cancer screening also, appropriate training programs may be expected to contribute to improvement of the lung nodule detection performance of radiological technologists on low-dose screening CT images.
The appropriate reading time for low-dose screening CT images has not yet been clarified; in general, the reading time depends on the experience of each radiologist, the number of CT slices per case, the number of abnormal findings on the CT images per case and the reporting system employed. In this study the technologists took approximately 1.3 times longer to read the CT images than the radiologists did, despite the smaller amount of input into the reading system by the technologists than by the radiologists. However, one study found that longer CTC interpretation times increased the accuracy of interpretation of the technologists .
One study reported the usage of a CAD system for mammogram reading by a radiologist and three technologists, and concluded the CAD system failed to improve performance of either the radiologist or the three technologists . Another study evaluated the efficacy of a CAD system for low-dose CT lung cancer screening, and found the detection accuracy improved significantly for thoracic and general radiologists using the CAD system; however, no statistically significant difference in the diagnostic accuracy obtained with and without the use of CAD was seen for residents . In the field of low-dose CT lung cancer screening, whether a CAD system will improve the lung nodule detection performance of radiological technologists must be investigated in the future.
In Japan, low-dose CT lung cancer screening has spread in medical check-up facilities as a screening option. In order to maintain the quality of CT screening, a non-profit organisation called the Accreditation Council for Lung Cancer CT Screening was established in 2007 . The Accreditation Council certified radiologists and technologists as specialists for low-dose CT lung cancer screening images after they attended training courses. Attending several lectures during a 5-year period is required for updating certification.
The limitations of this study were as follows. First, no receiver-operating characteristic analysis based on the confidence level of the localisation of the lung nodules was performed. The reason is that the 10 technologists were neither involved in low-dose CT lung cancer screening nor read data sets of low-dose TS-CT images consecutively and routinely, whereas the 11 radiologists read low-dose TS-CT images consecutively in the course of their routine work. In addition, the reading method of the technologists differed from that of the radiologists in this study, as described in the “Reading procedures” section above. Second, the reading sessions during this study were scheduled for weekends, when the observers were available. Although there was no time limit set for the reading, the tight schedule may have affected the observers' performances. Third, no data regarding measurements of the diameters of the nodules, classification of the consistency of the nodules or differentiation between benign and malignant nodules by the radiologists are reported here, because inclusion of such data was beyond the scope of this article. We propose to conduct a study including these data in the future.
In conclusion, well-trained technologists may contribute to the detection of lung nodules ≥5 mm in diameter representing pure and mixed GGOs, which are more likely to be malignant than solid nodules, on low-dose screening CT images.
The authors thank the radiological technologists who participated in this study: Hiroshi Kawasaki, Takeo Kunii, Shoji Makabe, Futoshi Matsunaga, Kenji Miyagi, Hiroshi Nagamatsu, Masafumi Shinozaki, Masaki Tamura, Yoshinobu Yajima and Mitsuo Wada. The authors thank the staff of the Screening Technology and Development Division in the Research Center for Cancer Prevention and Screening: Masahiro Suzuki, Chieko Nagashima, Hiromitsu Daisaki and Takeshi Murano.
This research was supported in part by a grant-in-aid from the Third-term Comprehensive Cancer Control Strategy sponsored by the Ministry of Health, Labor and Welfare, Tokyo, Japan (16-016), and a grant-in-aid from the Ministry of Health, Labor and Welfare, Tokyo, Japan (21-5-1).